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ABSTRACT 


Maintainability  demonstration  is  a  testing  procedure  for 
assuring  the  acquisition  of  equipment  and  systems  with  satisfac¬ 


tory  jc&lntainability  characteristics.  The  results  of  a  study  to 
improve  maintainability -demonstration  procedures  for  Air  Force 
equipment  are  presented  in  this  report. 


An  industry-  and  Government -wide  survey  was  conducted  to  pro¬ 
vide  insight  into  the  current  status  of  maintainability  demonstra 
tion  and  to  Initiate  resesirch  into  the  managerial,  administrative 
and  technical  aspects  of  demonstration.  Specific  recommendations 
and  guidelines  were  developed  on  the  following: 


Management  planning  for  maintainability  demonstration 


*  Maintainability-index  selection 

•  Maintenance -task  ssmpling  procedures 


•  Statistical  maintainability -demonstration  test  plans 


•  Test  administration  and  implementation 


The  use  of  prior  information  for  specifying  numerical  require 
ments,  designing;  statistical  sampling  procedures,  developing  test 
criteria,  and  applying  Bayesian  tests  was  also  investigated;  ap¬ 
plicable  procedures  and  data  are  included  in  this  report. 
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SECTION  T 


INTRO  Pi  OTION 


1.1  THE  CONCEPT  0 F  MAHTIA  UTILITY  DEMONSTRATION 

Maintainability  demonstration  is  the  process  by  which  a  cus¬ 
tomer  determines  if  a  product  he  intends  to  buy  will  exhibit  sat¬ 
isfactory  maintains]- ility  characteristics.  The  specific  approach 
used  can  range  from  reliance  on  the  producer's  assurance  and 
good  reputation  to  an  extensive  controlled  fie^d  test  of  the  prod¬ 
uct  .  A 


Neither  of  these  extremes  is  satisfactory.  The  producer's 
reputation  is  more  pertinent  to  selecting  him  initially,  and  his 
assurance  cannot  be  accepted  ’unless  there  are  adequate  factual 
data  to  support  it.  Often,  for  novel  or  complex  equipment,  it 
is  not  possible  to  assure  that  the  equipment  will  perform  satis¬ 
factorily  in  the  field  maintenance  environment  unless  some  form 
of  controlled  testing  is  performed.  On  the  other  hand,  extensive 
field  tests  are  generally  costly  and  time-consuming. 

The  current  Air  Force  policy  is  generally  to  perform  limited 
controlled  tests,  employing  standard  statistical  procedures  to 
determine  conformance  with  specified  maintainability  character¬ 
istics  . 

Two  military  standards  provide  the  requirement  and  direc¬ 
tion  for  conducting  a  maintainability-demonstration  test.  MIL- 
FTD-4Y0,  "Maintainability  Program  Requirements"  (Systems  a^d 
Equipments),  21  March  1966,  contains  the  following  detailed  re¬ 
quirement  for  such  a  ‘est: 

"Para.  r.ll  Demonstrate  Achievement  of  Maintain¬ 
ability  Requirements 

The  achievement  of  maintainability  requirements 
shall  be  demonstrated  as  specified  in  the  contract. 

The  demonstration  will  normally  be  accomplished  in  ac¬ 
cordance  with  MIL-STD-471,  'Maintainability  Demonstra¬ 
tion'.  which  includes  contractor  preparation  and  sub¬ 
mission  of  a  demonstration  plan  and  report  to  the 
procuring  activity.  The  demonstration  plan  must  be 
responsive  to  the  maintainability  program  established 
by  the  requirements  of  this  standard.  Maintainability 
demonstration  efforts  shall  be  integrated  with  other 
system  testing  requirements  3uch  as  proof  of  design, 
bread-board,  prototype,  environmental ,  production  and 
acceptance.  Maintainability  demonstration  data  will 
be  used  to  incrementally  verify  the  achievement,  of 
maintainability  design  requirements  and  to  update  the 
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maintainability  parameter  values  from  the  maintain¬ 
ability  analysea  and  predictions.  The  formal  main¬ 
tainability  demonstration  performed  to  determine  con¬ 
tract  compliance  shall  be  conducted  in  an  operational 
or  simulated  operational  environment  as  specified  in 
the  contract." 

The  referenced  standard,  MIL- STD-471,  "Maintainability 
Demonstration",  lp  February  1966,  Notice  1,  4  April  1968,  pro¬ 
vides  the  detailed  procedures  for  planning  and  conducting  main¬ 
tainability-demonstration  tests,  and  is  the  document  most  often 
invoked  in  current  contracts  that  require  such  tests. 

The  demonstration  procedure  is  essentially  a  statistical 
test  of  a  hypothesis.  In  this  case,  the  hypothesis  is  generally 
of  the  form  that  a  specified  maintainability  characteristic 
(e.g.,  mean  actlve-corrective-maintenance  time)  meets  a  speci¬ 
fied  numerical  value.  Accordingly,  the  standard  approach  has 
been  one  of  acceptance  sampling,  in  which  known  risks  of  wrong 
decisions  (rejecting  a  satisfactory  product  or  accepting  an  un¬ 
satisfactory  product)  are  considered  In  relation  to  aample  size 
(e.g.,  number  of  maintenance  actions  observed).  One  goal  is  to 
arrive  at  risk  levels  and  sample  requirements  that  meet  existing 
or  implied  constraints. 

The  statistical  nature  of  a  maintainability-demonstration 
test  imposes  requirements  on  several  aspects  of  the  test  pro¬ 
cedure,  including  the  tebt  environment,  tne  sampling  procedure, 
and  the  analysis  of  test  results.  Therefore,  a  demonstration 
t;est  cannot  be  evaluated  or  a  new  one  proposed  without  careful 
consideration  of  these  factors  as  they  relate  to  the  inferential 
nature  of  the  statistical  test. 

Therefore,  the  demonstration  procedure  can  be  viewed  as 
comprising  two  major  areas  —  the  planning,  management,  and  im¬ 
plementation  of  the  test;  and  the  statistical  procedures  to  be 
employed  in  the  decision-making  process. 

1.2  NUMMARY  OF  MAJOR  OBJECTIVES 

The  overall  objective  of  the  maintainability-demonstration 
phase  of  the  program  to  develop  maintainability  techniques  is  to 
develop  improved  procedures  for  planning,  implementing,  and  eval¬ 
uating  ma  “tainaoility-demonstration  tests.  Accordingly, 
effort  was  directed  at  both  the  managerial/administrative  aspects 
and  the  technical  and  statistical  aspects  of  the  demonstration 
element  In  the  maintainability-program  plan. 

Specific  study  was  devoted  to  the  following  subjects: 

‘  Management  of  the  maintainability-demonstration  pro¬ 
gram 


•  Prior  information  approaches  in  test  denign  and 
analysis 

•  Statistical  analysis  of  maintainability-demonstra¬ 
tion  test  results 

In  the  area  of  management  and  administration  of  the  main¬ 
tainability-demonstration  effort,  guidelines  and  specific  ap¬ 
proaches  were  developed  to  provide  assurance  that  (l)  the 
specified  maintainability  index  is  appropriate  and  realistic, 

(2)  the  task- sampling  proceau.?e  is  adequate,  (?,)  the  test  is 
conducted  in  an  unbiased  and  meaningful  manner,  and  (4)  the  re¬ 
sults  are  analyzed  and  interpreted  properly. 

The  use  of  prior  information,  such  as  previous  history  on 
similar  items,  results  of  previous  tests,  and  inputB  from  the 
maintainability -prediction  efforts,  were  incorporated  whenever 
possible  into  procedures  for  specification,  task  sampling,  and 
test  design.  In  particular,  new  Bayesian  approaches  for  main¬ 
tainability  demonstration  were  developed. 

In  this  report  the  statistical  basis  for  demonstration  is 
reviewed,  and  guidelines  for  selecting  general  approaches  (e.g., 
fixed  versus  sequential  sampling)  are  presented.  An  extensive 
set  of  statistical  procedures  for  demonstration,  offering  im¬ 
provements  in  rigor,  efficiency,  or  applicability  over  current 
procedures,  is  described.  Guidelines  are  presented  for  select¬ 
ing  the  appropriate  plan  to  fit  particular  circumstances. 


SECTION  II 


SURVEY  ON  MAINTAINABILITY  DEMONSTRATION 


2.1  INTRODUCTION 

Although  maintainability  demonstration  has  not  been  applied 
to  the  same  extent  as  reliability  demonstration,  it  was  believed 
that  enough  experience  has  been  accumulated  to  warrant  a  survey 
of  industry  and  Government  personnel  concerned  with  it.  Accord¬ 
ingly,  a  conprehennive  questionnaire  was  developed  to  cover  both 
management  and  technical  aspects  of  maintainability  demonstra¬ 
tion.  Exhibit  1  is  a  copy  of  this  questionnaire.  A  rather  com¬ 
prehensive  discussion  of  the  results  of  this  survey  is  presented 
here  because  these  results  provide  a  good  summary  of  the  current 
status  of  maintainability  demonstration.  Particular  attention  is 
given  to  problems  that  the  respondents  believe  have  not  yet  been 
solved  by  current  procedures. 

2.2  RESPONDENTS 

This  questionnaire  was  mailed  to  approximately  200  people 
and  agencies  actively  engaged  in  the  field  of  mr.intc  inability 
management  and  engineering.  Pour  major  sources  were  used  to 
obtain  respondents: 

*  The  EIA-G-42  maintainability  committee  mailing  list 

*  The  military  and  Government  organizations  and  personnel 
involved  in  MIL- STD-471  coordination 

*  Organizations  that  deal  with  the  RADC  Maintainability 
Section 

•>  A  list  of  authors  on  maintainability,  obtained  through  a 
literature  search  of  various  technical  journals  and  sym¬ 
posium  proceedings 

2.3  SUMMARY  OF  RESULTS 

A  total  of  39  usable  replies  was  received  (approximately  a 
2b-percent  return),  some  of  which  contained  detailed  comments  on 
one  or  more  aspects.  Table  I  presents  a  summary  tabulation  of 
the  responses  received. 

2.4  DISCUSSION  OF  RESULTS 

The  results  of  the  survey  are  discussed  in  this  subsection. 
A  summary  of  the  most  pertinent  statistics  of  Table  I  is  pre¬ 
sented  and  interpreted.  Indications  of  how  Improvements  can  and 
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should  he  made  are  also  presented,  but  detailed  consideration  of 
such  means  is  reserved  for  later  subsections.  In  a  sense,  then, 
the  discussions  of  the  survey  questions  serve  as  a  summary  of 
some  of  the  major  areas  to  be  considered  in  subsequent  sections 
of  this  report. 

2.4.1  Respondent  Background 

Of  the  39  responses  received,  approximately  75  percent  rep- 
resented  industry  viewpoints;  approximately  75  percent  of  the 
respondents  have  been  involved  with  one  or  more  maintainability 
demonstrations,  the  total  of  such  demonstrations  exceeding  40. 

The  most  commonly  used  tests  were  those  of  MIL-M23313(9) 
and  MIL-M26512(8),  which  correspond  to  Test  Methods  3  (4  appli¬ 
cations)  and  2  (9  applications),  respectively,  of  MIL-STD-471. 
Thus,  of  4l  identified  tests  used  for  demonstration,  17  demon¬ 
strations  were  based  on  the  procedure  of  Test  Method  2,  MIL-STD- 
471,  and  13  demonstrations  were  based  on  the  procedure  of  Test 
Method  3,  MIL-STD-471.  The  only  other  significant  applications 
were  the  five  demonstrations  based  on  Test  Method  1  of  MIL-STD- 
471. 


Comments  on  the  statistical  aspects  of  the  procedures  used 
thus  apply  generally  to  MIL-STD-471,  Test  Methods  1,  2,  and  3. 
General  comments  on  management  and  administrative  aspects  of 
Government  maintainability-demonstration  standards  pertain 
equally  to  the  older  MIL-M26512  and  MIL-M23313  standards. 

2.4.2  Rejection  Experience 

Seven  rejections  were  reported  by  the  respondents.  In  only 
one  case,  however,  was  a  retest  performed  —  a  step  that  Is  rec¬ 
ommended  in  the  maintainaPility-demonstration-plan  provisions  of 
MIL-STD-471 .  In  th.ee  cases,  the  test  was  extended  —  a  proce¬ 
dure  that  would  normally  result  in  exceeding  the  designed  test 
risks.  In  two  canes,  the  applicable  requirement  was  waived. 

Since  controlling  maintainability  and  providing  information 
for  evaluation  purposes  are  major  purposes  of  demonstration,  it 
is  necessary  to  act  on  rejections  in  a  positive  manner.  Cor¬ 
recting  maintainability  design  to  eliminate  causes  of  rejection 
and  then  retesting  would  seem  to  be  the  most  prudent  course.  A 
slagle  test  extension  or  waiver  of  the  requirement  avoids  the 
central  Issue. 

2.4.3  Purpose  of  Maintainability  Demonstration 

Almost  half  of  the  respondents  indicated  that  control  of 
achieved  maintainability  was  the  major  purpose  of  maintainability 
demonstration.  One-third  believed  that  the  Information  provided 
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w  JsnemjuiE  on  mamtamasiuty  rcwssrtATiaN 


Please  Return  to: 


information  Concerning  the  Respondent 


Company  or  Agency 
Division  or  Department 


Address 


Nature  of  Business 


ARINC  Research  Corporation 
a  Subsidiary  of  Aeronautical  Radio,  Xnc. 

2551  Rlva  Road 
Annapolis,  Maryland  21401 
Attention:  H.S.  Balaban 

EXHIBIT  1 

QUESTIONKAIFiS  MAILED  TO  MAINTAINABILITY 
MANAGEMENT  AND  ENGINEERING  PERSONNEL 


,v 


verify  M  prediction 


EXHIBIT  1  (continued] 
9 


S  SI  S  £ 


Iptiiiiusj  ttgumsMiuti  Iff)  u!«s»m»iMi»N  it  situs* 


SUMMARY  OF  R8SPOWSS5  TO  MAZBTMlSA&XLXTx  -XuDCHStFATlOH  PtlffiTIOmm 


Industry  29 

Mllitary/Government  10 


Involved  with  on*  or  ear*  SO 
No  Involvement  7 
So  Answer  2 


Designer  of  Equipment 
Designer  of  Maintainability- 
Demonstration  Test 
Conductor  of  Malntalnablllty- 
Denonatr&tlon  Test 
Monitor  of  Malntaiuabillty- 
Deraonstratlon  Test 


Test  extended 
Equipment  Redesigned 
Bqulpaent  Redesigned  end  Retested 
Applicable  Requirement  Halved 
Penalty  Provision  Invoiced 
Other 


MIL- STD-471 

Plan  1  £ 
Plan  2  S 
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Element  Satisfied 

unsatisfied 

Definition  of  Terms 

39 

37 

Test  Condition.  Requirements 

37 

30 

Support  Material  Requirements 

44 

22 

Test  Personnel  Requirements 

35 

28 

Teat  Administration  and  Reporting 

Requirements 

45 

23 

Use  of  Prior  Information 

Parameter  Specification 

12 

Distribution  Analyses 

7 

S&qple  Selection 

24 

Bayesian  Tests 

4 

Other 

2 

No  Response 

5 

' 
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Intermittent  failures 

Multiple  failures 

tack  of  supporting  Materials 
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Kxtsnslve  resTurcee  required 

Other 

So  Response 


Historical  Data 
Allocation 
Contractor  Values 

Preliminary  MMlntalnab lllty  Predictions 
Other 


eclflcatlon 
Jtedian  and  Percentile 
Median  and  Variance 
Mean,  Median,  Percentile 
Ms  an.  Median,  Variance 


Mean 

Median 

Percentile 

Mean  .irvd  Median 

Mean  and  Percentile 

Mean  and  Vsrlance 


Total  Mean 
Total  Median 
Total  Percentile 
Total.  Vsrlance 


A.  Fixed  Sample 
Multiple  Sasple 
Sequential 

B.  Faraastric 
Baoparanttric 

C.  Classical 
Bayesian 
Decision  Theory 

B.  Simlated  Failures 
Actual  Failures 
Cosfciaation 
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is  most  important.  These  two  answers ,  of  course,  are  not  mutu¬ 
ally  exclusive;  i.e„,  it  can  be  assumed  that  the  Information  can 
and  will  be  used  to  control  maintainability. 

Sight  respondents  believed  that  maintainability  demonstra¬ 
tion  is  most  useful  for  verifying  a  maintainability  prediction. 

In  one  sense,  this  would  imply  that  the  prediction  is  more  per¬ 
tinent  to  the  maintainability  program  than  actual  test  results. 
This  would  be  the  case,  for  exanple,  If  the  field  maintenance  and 
logistic  planning  depended  heavily  on  the  maintainability  predic¬ 
tion. 

In  practice,  however,  the  maintainability  demonstration  is 
geared  to  the  contractual  maintainability  requirements,  and  the 
prediction  is  another  element  of  a  maintainability  program  to 
provide  control.  If  a  prediction  indicated  that  the  contractual 
requirement  would  not  be  met,  it  would  be  unwise  to  proceed  to 
the  demonstration  without  carefully  viewing  the  requirement,  the 
design,  and  the  prediction  procedure  and  inputs  to  make  any  nec¬ 
essary  revisions. 

2.4.4  Views  on  Aspects  of  Maintainability  Standards  or  Contracts 

As  can  be  seen  in  Table  I,  the  percentage  of  respondents  who 
were  satisfied  with  the  maintainability  aspects  listed  in  the 
questionnaire  is  not  encouraging.  Almost  half  of  the  responses 
indicated  that  on®  or  more  contracts  or  standards  had  unsatis¬ 
factory  definitions.  Test-condition  requirements  and  test- 
personnel  requirements  also  received  relatively  unsatisfactory 
ratings.  While  support  material  and  test-administration  and 
-reporting  requirements  were  rated  better  than  the  others,  about 
one-third  of  the  ratings  were  unsatisfactory. 

These  results  vividly  illustrate  the  need  for  careful  atten¬ 
tion  to  the  specifics  of  definitions  and  requirements  concerning 
maintainability  demonstration  in  contractual  documents  and  in 
associated  military  standards. 

2.4.5  Use  of  Prior  Information 

As  would  be  expected,  the  two  most  prominent  categories 
given  for  the  use  of  prior  information  in  maintainability  demon¬ 
stration  were  cample  selection  and  parameter  specification.  Cur¬ 
rent  sampling  procedures  are  usually  a  forcrof  proportional 
stratified  sampling,  which  requires  identification  of  tasks, 
means  for  grouping  tasks,  and  relative  frequency  of  occurrence 
of  the  groups.  Parameter  specification  may  require  information 
m  (1)  a  higher -level  requirement  such  as  availability,  (2)  the 
type  of  malnie  nce-tiaa  distribution,  and  (3)  achieved  maintain¬ 
ability  on  similar-  systems  or  equipments. 
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2.4.6  Difficulties  Experienced 

The  two  difficulties  mentioned  assist  often  in  the  survey  were 
differences  between  the  test  environment  and  the  field  environ¬ 
ment,  and  the  extensive  resources  required  for  planning  and  con¬ 
ducting  the  demonstration. 

Differences  between  test  and  field  environments  will  always 
be  present,  but  every  effort  should  be  made  to  mini wise  them 
through  careful  planning.  On  the  other  hand,  there  may  be  eases 
In  which  duplication  of  environment  will  be  costly  and  difficult. 

A  compromise  approach  is  to  know  what  the  environmental 
differences  are  and  to  adjust  the  specified  parameter  values 
accordingly. 

If  the  maintainability  demonstration  is  considered  merely  an 
exercise  in  statistics,  with  little  or  no  enforcement,  the  expen¬ 
diture  of  funds  is  essentially  wasteful.  If  the  contractor  knows 
that  failure  to  pass  the  demonstration  test  will  require  further 
efforts  on  his  part  to  improve  the  maintainability  design,  he 
will  have  the  incentive  to  provide  carefully  far  the  maintain¬ 
ability  demonstration. 

Economy,  of  course,  is  still  a  major  objective}  careful 
planning  and  integration  with  the  complete  test  program  is  essen¬ 
tial.  Sample  sizes  can  be  reduced  by  using  the  most  efficient 
tests  or  by  accepting  higher  risks.  Use  of  prior  information  in 
a  Bayesian  test  is  one  new  approach  to  limiting  the  amount  of 
necessary  testing. 

2.4.7  Opinions  on  Specified  Maintainability  Values 

Almost  6o  percent  of  the  respondents  believed  that  specified 
maintainability  values  were  realistic.  It  was  noted,  however, 
that  military  and  Government  personnel  were  less  convinced  of 
this  realism  (45  percent)  tmn  Industry  personnel  (63  percent). 
Generally,  it  should  be  the  customer  (l.e. ,  military)  who  speci¬ 
fies  quantitative  values,  but  In  practice  the  contractor  may 
play  a  prominent  role  In  such  specifications. 

2.4.8  Approach  to  Maintainability  Specifications 

Four  alternative  approaches  to  specifying  maintainability 
values  were  presented,  and  approximately  44  percent  of  the 
respondents  favored  an  approach  based  on  an  allocation  of  a 
higher-level  requirement  such  as  availability  or  effectiveness* 
Since  most  of  the  respondents  were  from  Industry,  this  might  be 
a  reflection  of  their  desire  for  flexibility,  which  might  be 
stated  as  follows*  "Tell  us  your  overall  objective}  let  us  worry 
about  how  we  will  achieve  It”* 
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In  many  eases,  this  aright  be  a  prudent  policy,  but  there  are 
situations  in  which  complete  freedom  is  undesirable.  For  exam¬ 
ple,  the  following  sets  of  MEBF  and  MPTR  values  will  both  yield 
the  same  steady-state  availability  of  0.90: 

•  Set  1:  MTBF  9;  KEZR  1 

‘  Set  2:  MEBF  90;  MTTR  10 

The  cost  of  the  two  sets,  however,  can  differ  greatly.  The 
first  set  requires,  on  the  average,  10  times  the  number  of  main¬ 
tenance  actions  as  the  second  for  continuously  operated  systems; 
and  if  maintenance  consists  generally  of  replacement,  3et  1  will 
require  much  greater  inventory  levels.  On  the  other  hand,  if 
mission  time  is  small,  for  example,  less  than  one  hour,  a  9-hour 
MEBF  may  be  acceptable,  but  a  10- hour  downtime  may  be  operation¬ 
ally  unacceptable  from  the  viewpoint  of  readiness.  If  the  sys¬ 
tem  is  not  operated  between  missions.  Set  1  may  be  preferable. 

Examples  such  as  these  emphasize  the  need  for  as  complete  a 
specification  ss  possible  for  the  overall  requirement  and  asso¬ 
ciated  constraints.  The  application  of  an  allocation  procedure 
under  these  conditions  is  generally  a  good  policy,  because  it 
provides  assurance  that  requirements  will  be  consistent  with  the 
overall  objective. 

The  second  most  frequently  mentioned  approach  to  maintain¬ 
ability  specification  was  the  use  of  historical  data.  This  is 
a  natural  means  for  imposing  realistic  requirements.  It  is  gen¬ 
erally  unwise  to  specify  a  maintainability  value  without  refer¬ 
ence  to  the  state  of  the  art. 

Of  course,  for  new  equipments,  maintenance  philosophies, 
and  operational  procedures,  there  may  be  little  relevance  to  past 
history,  and  it  is  in  such  cases  that  problems  arise.  In  these 
situations,  conservatism  may  be  the  most  prudent  course,  because 
the  risk  of  failure  is  too  great  if  large  inprovements  over  the 
state  of  the  art  are  expected. 

It  may  also  be  possible  tc  provide  some  flexibility  in  the 
requirement  so  that  as  development  progresses  and  exploratory- 
test  results  are  evaluated,  a  realistic  maintainability  require¬ 
ment  for  demonstration  can  then  be  invoked  upon  agreement  between 
contractor  and  customer,  duch  a  policy  will  require  some  term  of 
restriction  in  the  initial  contract  (e.g.,  minimum  and  maximum 
values)  to  protect  both  parties. 

A  third  approach  for  new  types  of  systems  and  maintain¬ 
ability  policies  is  an  Incentive  arrangement  whereby  only  a  mini¬ 
mum  maintainability  value  is  specified.  The  demonstration 
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procedure  may  then  be  an  estimation  process  in  which  the  incen¬ 
tive  payment  or  penal ty  is  geared  to  the  estimate  (either  point 
or  interval) , 

2.4.9  Lognoriaal-Paraceter  Specification 

To  the  question,  which  parameter  or  parameter  conciliation 
should  be  specified  if  maintenance  times  were  distributed  log- 
normally,  the  most  frequent  answer  (10  of  36  responses)  was  the 
mean/Variance  conciliation.  This  is  ‘surprising  because  (1)  no 
currently  used  method  is  based  on  this  combination;  and  (2)  the 
variance  of  a  lognormal  distribution,  unlike  the  median  and  per¬ 
centile  values,  is  difficult  to  interpret. 

Of  the  two  central-tendency  parameters,  the  mean  was  includ¬ 
ed  In  26  responses  and  the  median  in  l4.  This  Is  In  agreement 
with  the  response  to  the  previous  question,  in  which  allocation 
was  the  preferred  approach  for  specification,  since  the  mean 
possesses  more  desirable  properties  than  the  median  for  alloca¬ 
tion. 


The  mean/percentile  coabinatipn  was  the  next  preferred 
specification,  and  this  combination  corresponds  to  several 
MEL-3TD-471  plans. 

Three  responses  referred  to  three  lognormal  parameters,  but 
since  any  two  of  the  three  parameters  listed  dictate  the  value 
of  the  others,  the  three-parameter  specifications  have  little 
justification. 

2.4.10  Type  of  Statistical  Test 

Prom  the  four  categories  considered,  the  most  preferred 
plan  (assuming  the  responses  were  independent  for  each  category) 
would  be  a  classical,  sequential  nonparametric  test  based  on  a 
combination  of  simulated  and  naturally  occurring  failures.  Kb 
currently  used  test  procedure  is  a  nonparametric  sequential  pro¬ 
cedure,  although  Test  Method  1  can  be  considered  such  a  test 
since  the  lognormal  assumption  is  used  primarily  for  converting 
the  lognormal-p&rameter  specification  to  a  binomial -parameter 
specification. 

The  nonparametric  and  classical- type  tests  sirs  the  clear 
choices  for  the  alternatives  presented.  The  sequential  test  is 
only  slightly  preferred  over  the  fixed-sanple-size  test,  while 
the  use  of  s inflated  failures  only  is  almost  as  preferred  as  a 
combination  of  simulated  and  actual  failures. 

2.4.11  Views' on  MIL-STD-471 


All  plans  received  relatively  good  ratings  on  the  three  as¬ 
pects  listed.  There  was  surprisingly  little  comment  on  the  risk 


errors  in  the  MXL-STD-471  methods.  Many  comments  were  related  to 
(1)  the  inflexibility  of  some  of  the  methods,  e.g.,  a  fixed  sample 
size  for  methods  3  and  4;  (2)  inappropriateness  of  the  lognormal 
assumption  for  specific  types  of  equipments;  and  (3)  the  need  for 
large  sanple  sizes. 

The  comment  about  inflexibility  is  valid  since  the  MIL-STD- 
471  plans  do  not  generally  provide  for  varying  risks.  The  log- 
norsal  assunption  may  not  nold  for  a  specific  type  of  system  or 
maintenance  comment,  and  that  is  why  a  nonpar ametric  test  is  in¬ 
cluded  in  the  standard.  However,  other  nonparametric  tests  are 
available  that  would  offer  more  flexibility. 

The  sanple  size  (e.g.,  f>0  for  Test  Method  4)  is  based  solely 
on  tha  risks.  The  only  way  to  reduce  the  sample  size  for  a  given 
procedure  is  to  increase  risks  (or  effectively  have  less  dis¬ 
crimination  in  the  test  between  acceptable  and  unacceptable 
maintainability).  The  use  of  more  efficient  test  procedures  — 
sequential  rather  than  fixed,  parametric  rather  than  nonpara¬ 
metric,  and  Bayesian  rather  than  classical  tests  —  is  another 
alternative  that  can  lead  to  reduced  sanple  sizes. 

2.4.12  Approval  of  Stratified  Sampling 

Twenty-seven  of  37  responses  indicated  approval  of  strati¬ 
fied  sanpling  procedures.  There  were,  however,  many  comments  on 
stratification,  with  many  different  suggestions  for  improving 
current  task- select ion  procedures.  Very  little  comment  was  made 
on  the  current  use  of  analytical  procedures  based  on  simple  ran¬ 
dom  sampling  when,  in  fact,  the  sanple  observations  are  usually 
obtained  through  a  stratified  procedure. 

The  reason  most  often  given  for  favoring  stratified  sanpling 
was  that  it  is  the  best  way  to  ensure  representativeness. 
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SECTION  III 


DEMONSTRATION  AS  A  RAINTAINABILITY -PROGRAM  ELEMENT 
3.1  GENERAL 

In  Section  I,  the  requirement  for  demonstration  as  a 
maintainability-program  element  as  specified  in  MIL-STD-470  was 
noted.  The  major  points  of  paragraph  5-H  of  MIL-STD-470  are 
as  follows: 

(1)  Maintainability  demonstration  is  a  contractual  re¬ 
quirement  . 

(2)  Maintainability  demonstration  will  normally  be  per¬ 
formed  in  accordance  with  MIL- STD-471. 

(3)  The  contractor  will  submit  a  maintainability-demon¬ 
stration  plan. 

(4)  The  maintainability-demonstration  test  will  be  inte- 
£T*nted  with  other  system  requirements. 

(5)  The  formal  demonstration  will  be  conducted  in  an  opera 
tional  or  simulated  environment. 

With  regard  to  item  2,  MIL- STD-471  contains  the  detailed 
procedure,  but  of  particular  pertinence  to  this  section  are  the 
major  elements  to  be  included  in  the  maintainability-demonstra¬ 
tion  plan,  which  are  listed  in  Table  II. 


TABLE  II 

MAJOR  ELEMENTS  OP  A  MAINTAINABILITY  - 
DEMONSTRATION  PLAN  (MIL- STD-471) 

Description  of  Demonstration  Conditions 

Description  of  Test  Team 

Description  of  Demonstration  Support  Material 
Predsmonetrat ion -Phase  Schedule 
Description  of  Formal  Demonstration  Test 
Retest-Phase  Schedule 
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Timely  planning  and  careful  management  of  the  maintaina¬ 
bility  demonstration  is  required  to  fulfill  the  requirements  of 
IHL-8TD-470  and  471.  This  chapter  presentn  a  general  discussion 
of  some  of  the  major  problems  that  can  occur  in  such  planning 
and  management  and  offers  guidelines  for  selecting  and  imple¬ 
menting  the  best  policies. 

3.2  MANAGEMENT  RESPONSIBILITIES  IN  MAINTAINABILITY  TESTING 
3.2.1  Development -Test  and  Demonstration-Test  Management 


The  maintainability-demonstration  procedure  is  generally 
the  last  of  a  series  of  tests  and  evaluations  concerned  with 
maintainability  parameters.  Each  of  these  tests  and  evaluations 
is  designed  to  provide  inlormation  about  system  maintainability. 
Tests  designed  to  aid  in  improving  or  achieving  desirable  main¬ 
tenance  characteristics  can  be  classified  as  information  tests 
or  development  tests.  The  management  aspects  of  development 
testa  and  maintainability-demonstration  tests  are  summarized 
in  Tables  III  and  IV,  respectively. 

3.2.2,  Overall  Test  Planning 


It  Is  the  re3ponsib5 lity  of  management  to  plan  and  manage 
the  necessary  tests  In  such  a  manner  that  pertinent  and  timely 
information  concerning  maintainability  achievement  will  be  pro¬ 
vided  as  economically  as  possible.  This  is  not  an  easy  taBk  in 
view  of  the  many  unknowns  that  exist  In  a  typical  system-develop¬ 
ment  program. 


An  overall  test  plan  should  be  developed  at  the  beginning 
of  the  program.  At  this  point,  the  plan  will  not  be  completely 
detailed.  However ,  it  should  provide  for  dotermining  specific 
test  requirements  and  procedures  at  some  point  before  the  testing 
Is  to  be  performed,  allowing  the  review  of  all  aspects  of  the 
tests  by  interested  design,  engineering,  and  management  groups. 


Since  the  features  of  many  types  of  tests,  particularly 
those  in  the  development  stage,  will  depend  on  the  achievements 
made  and  problems  encountered  during  the  design-evaluation  pro¬ 
cess,  the  test  program  must  be  flexible  enough  to  allow  appro¬ 
priate  changes  in  test  parameters,  procedures,  and  decision 
rules.  Such  changes,  however,  should  be  carefully  controlled 
and  reviewed  by  the  cognizant  Air  Force  agency.  Table  V  sum¬ 
marizes  tne  steps  necessary  for  overall  test  planning. 
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The  decision  to  require  a  formal  maintainability-demon¬ 
stration  test  rests  with  the  Air  Force  and  should  be  based  on 
tactical  considerations,  mission  requirements,  cost  of  tests. 


MANAGEMENT  ASPECTS  OP  DEVELOPMENT  TESTS 


Purpose  of  Tests 

To  determine  physical  realizability,  to  determine  funotior&l 
capabilities,  to  establish  the  bade  design. 

Responsible  Groups 

Air  Force  or  contractor's  design-engineering  group,  with 
support  from  other  groups  as  required. 

General  Description 

Development  teats  are  usually  informal  exploratory  testa 
designed  to  provide  fundamental  R&D  information  about  a 
basic  design.  Nominal  environmental  levels  are  used  unless 
the  test  is  specifically  oriented  to  check  for  effects  at 
environmental  extremes.  Sample  sites  are  limit  ad,  but  the 
general  principles  of  good  experimental  and  statistical 
design  should  be  followed. 

Examples  of  Specific  Types  of  Tests 

•  Component  Accessibility  *  Harmonization  Requirements 

•  Fault. -Isolation  Routines  •  Compatibility  Tests 

•  Test-Point  Adequacy 

Test  Scheduling 

Not  usually  specified  formally.  Design-engineering  group 
establishes  schedules  to  meet  design-development  objectives. 
Such  schedules  must  conform  to  development-program  mile¬ 
stones. 

Test  Items 

Basic  materials,  off-the-shelf  parts  and  assemblies,  proto¬ 
type  hardware. 

Test  Documentation 

Engineering  test  reports  and  analyses.  Maintainability 
information  to  be  documented  for  later  use  in  prediction, 
evaluation,  and  testing  tasks. 

Test  Follow-Up  Action 

Determination  of  design  feasibility  or  need  for  redesign. 
Implementation  of  test  information  in  further  design  work. 
Approval,  modification,  or  disapproval  of  design,  materials, 
and  parte. 

Maintainability  Provisions 

Proposed  materials  and  designs  to  yield  acceptable  maintain¬ 
ability  performance  are  tested  on  limited  eamples.  Packag¬ 
ing  tests  and  component-interaction  tests  are  examples.  All 
maintainability  data  should  be  fully  documented  for  future 
use  in  prediction,  assessment,  and  later  testing  activities 
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Purpose  or  Tests 

To  demonstrate  formally  that  the  maintainability  require¬ 
ment  s  are  aehelved. 

Responsible  Groups 

Air  Force  or  contractor's  program-rinagement,  maintain¬ 
ability,  and  effectiveness-assurance  grow*;  with  teste 
monitored  and  approved  by  the  Air  Force. 

General  Description 

Demon strati  a  tests  sre  performed  on  the  major  end  items, 
often  at  thw  highest  system  level,  under  realistic  opera¬ 
tional  and  environmental  conditions,  Rules  are  specif led 
for  classifying  failures,  performing  repairs,  allowing  de¬ 
sign  changes,  etc.  Time  is  an  inherent  test  parameter. 

The  test  design  is  usually  directed  towards  providing  a 
specified  statistical  confidence  for  making  an  appropriate 
decision . 

Test  Scheduling 

Demonstration- test  schedules  are  normally  contraot-specl- 
fied.  They  generally  occur  before  full-scale  production 
but  after  initial  production,  when  teat  samples  are  avail¬ 
able. 


Test  Items 

Production  hardware  at  major  end-item  level. 

Test  Documentation 

Contract- specified  procedures  or  clause  requiring  contrac¬ 
tor  to  submit  complete  test  plan.  Test  results  fully  docu¬ 
mented,  including  analyses  and  conclusions  concerning  the 
meeting  of  contract  requirements. 

Test  Follow-Up  Actions 

Acceptance  or  rejection  of  equipment  with  respect  to  'viin- 
talnability  requireaants .  Failure  to  pass  demonstration 
tests  will  require  appropriate  design  and  assurance  efforts 
on  the  part  of  the  contractor . 

Main  .inability  Provisions 


Demonstration  tests  are  specifically  designed  to  test  for 
maintainability  and  essoclatsd  parameters  at  the  equipment 
end  system  levels. 


f 


TABLE  V 

STEPS  XH  OWBALL  TEST  FLW3SDW 


1.  Determine  teat  requirement*  and  objective! 

2.  Review  existing  data  to  determine  if  any  existing  require¬ 
ment  a  can  be  met  without  teats 


3.  Review  a  preliminary  list  of  planned  teat*  to  determine 
whether  economies  can  be  realised  by  combining  individual 
test  requirements 

-■ 

4.  .Determine  the  necessary  testa 

S'  -"X 

3.  Allocate  time,  funds,  and  effort  to  perform  theae  testa 

6.  Develop  teat  specifications  at  an  appropriate  level,  or 
make  reference  to  applicable  sections  of  the  system  zpeeifl- 
ration  to  provide  direction  for  later  development  of  teat 
specifications 

7.  Assign  responsibility  for  test  "onduct,  monitoring,  analysis, 
and  Integration 

8.  Develop  review  and  approval  policies  for  test-reporting  pro¬ 
cedures  and  forms 


9.  Develop  procedures  for  maintaining  test-status  information 
throughout  the  entire  program 


and  an  evaluation  rf  the  likelihood  of  achieving  the  maintain¬ 
ability  requirement  simply  as  a  result  of  good  design  procedures 
without  a  demonstration  test  It  la  essential  to  recognize 
that  a  maintainability-demonstration  test  does  not  guarantee 
achieving  the  required  maintainability  It  focuses  the  con¬ 
tractor's  attention  on  maintainability,  but  often  this  la  not 
sufficient  unless  penalties  for  test  failure  are  Included  in 
the  contract. 


3-3.1 


There  are  no  fixed  rules  for  determining  Air  Force  and 
contractor  responsibilities  In  preparing  malntalnsblllty-demon- 
atratlon-test  plana.  The  peculiarities  of  Individual  procure¬ 
ments  require  flexibility  In  the  requirements  for  supplying 
information.  mL-STD-471  outline*  certain  requirements  for  a 
maintainability-teat  plan.  Generally  the  plan  ia  submitted  as 
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part  of  the  contractor's  proposal  and  Includes  details  that  are 
applicable  to  maintainability  information  supplied  by  the  Air 
Force.  The  specific  information  supplied  to  the  contractor  by 
the  Air  Force  should  meet  both  of  the  following  criteria: 

•  It  should  provide  direction  for  developing  the  test  de¬ 
sign. 

•  It  should  be  based  on  operational  or  tactical  constraints 
or  on  a  trade-off  analysis. 

In  most  cases,  these  criteria  will  be  met  if  the  following  are 
provided: 

'  Equipment  configuration  for  test 

•  Maintenance  concept 

•  Maintenance  environment 

•  Levels  of  mainte^^  r«vv  *o  be  demonstrated 

•  Modes  of  operation  for  the  test 

•  Test  team  organization 


3.3.2  Contractor 's  Responsibility 


In  his  proposal,  the  contractor  must  include  certain  infor¬ 
mation  about  the  maintainability-demonstration-test  plan,  re¬ 
gardless  of  whether  the  information  has  been  derived  from  the 
Air  Force  or  the  contractor.  The  plan  submitted  by  the  contrac¬ 
tor  to  the  Air  Force  should  include  the  elements  listed  in 
Table  VI,  which  was  extracted  from  MIL- STD- 471 . 


The  major  milestones  in  the  development  of  a  final  main¬ 
tainability-demonstration  plan  are  listed  in  Table  VII,  in  which 
it  can  be  seen  that  the  maintainability-demonstration  plan  must 
be  continually  updated  as  system  development  progresses,  to  re¬ 
flect  changes  in  requirements  and  design  and  to  Incorporate  the 
results  of  the  maintainability-program  design  reviews,  predic¬ 
tions,  and  assessments. 


Specific  dates  for  review  by  the  procuring  activity  should 
be  established  ?t  the  time  the  contract  is  awarded.  Before  the 
test  Is  conducted;  the  maintainability-demonstration  plan  and 
detailed  procedures  must  receive  final  approval  of  the  review¬ 
ing  activity. 


BUSHSMIS  OF  CX>RTEAOTORJS  K&imiEABniTy-B^iOHSiimfSOH  TEST 


Description  of  Demonstration  Conditions 

Quantitative  maintainability  requirement o 
Maintenance  concept 

Halntalnabilityodemonstration  environment 

Levels  of  maintenace  to  be  deoonstreted 

Demonstration  sites  and  facility  requirements 

Participating  agencies 

Mode  of  operation  for  the  test 

Item(s)  to  be  demonstrated 

Description  of  Test' Team 

Organization 

Degree  of  participation  for  contractor  and  procuring 
activity 

Assignment  of  specific  responsibilities 

Qualification,  quantity,  and  training  of  test-team  personnel 

Description’  of  Demonstration  Support  Material 

Support  equipment 
Tools  and  test  equipment 
Technical  publications 
Spares  and  consumables 
Safety  equipment 

Calibration  support  requirements 

! 

Predemon at rat ion- Phase  Schedule 


Assembly  of  test  tear* 

Training 

Preparation  of  facilities  and  support  material 

Description  of  Formal  Demonstration  Test 

Test  objectives 
Schedule  of  tests 
Task- selection  method 
Test  method 

Data -acquisition  method 
Analytical  and  calculation  methods 
Specific  data  elements 
Time  units  of  measurement 
Type  and  schedule  of  report* 

Description  and  schedule  of  preventive-maintenance  tasks 
Description  of  corrective-maintenance  tasks 

Retest  Phase 


TABLE  VTI 


MILESTONES  IN  THE  DEVELOPMENT  OF  THE 
»U;NTAINA3ILITY- DEMONSTRATION  PIAN  AND  PROCEDURE 


Period 

Input 

Output  for  Maintainability- 
Demonstration  Applications 

Prepropooal  and  Proposal 

Initial  work  state¬ 
ment 

Proposed  maintainability- 
demonstration  plan 

Ccr/crsct  Award 

Pinal  technical  and 
coat  requirements 

Updating  of  maintainability- 
demonstration  plan 

System  Development 

Maintainability 
design;  development- 
teat  results;  main¬ 
tainability  predic¬ 
tions  and  analyses 

Revisions  to  maintainability 
demonstration  plan 

Pinal  Test  Planning 

Overall  system-test 
program 

Final  maintainability-demon¬ 
stration  plan;  integration 
with  overall  teat  program 

Predemonstration  Phase 

Maintainability- 
demonstration  plan 

TeBt  team,  facilities,  and 
support  material  assembled; 
detailed  procedural  methods 
for  sampling,  analyzing,  and 
reporting  results  completed 

Maintainability  Demon¬ 
stration 

Approved  maintain¬ 
ability-demonstra¬ 
tion  plan  and  pro¬ 
cedures;  equipment, 
facilities,  material 
and  personal 

Results  of  maintainability- 
demonstration  teats 

Retest  Phase 
{If  necessary) 

Results  of  maintain¬ 
ability-demonstra¬ 
tion  tests;  retest 
plan 

Results  of  retest 
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Although  the  baalo  purpose  of  maintainability  demonstration 
Is  control,  AFJ3JC  Research  Corporation  has  found,  during  the  ~ * 
course  of  several  deaonstrationHBonltoring  studies,  that  m  im¬ 
portant  by-product  la  the  discovery  of  faults  and  procedures  that 
degrade  an  equipment's  reliability  and  maintainability. 

The  cc'^aents  in  Table  VXH  were  extracted  frost  several  AR23KS 
Research  lev  ter  reports  to  military  customer*  presenting  the  cb-  . 
aervatlone  of  AHIKC  Research  personnel  monitoring  maintainability 
demonstrations.  It  can  be  seen  from  these  c  ssets  that  the 
maintainability  demonstration  dan  provide  information  heading  to 
improvements  in  manuals  and  support  equipment,  specific  Circuit 
design,  overall  maintenance  -da sign,  philosophy,  packaging,  and 
sparing.  Because  of  this  desirable  by-product  of  the  Maintain¬ 
ability-demonstration  effort,  the  oustoaer  should  plan  to  have 
knowledgeable  representatives  monitoring  the  test  to  discover 
design  deficiencies  and  recommend  means  for  liqprovement* 

Contractual  allowance  for  remedying  serious  deficiencies 
shotud  also  be  considered.  A  specific  maintenance  design  or 
procedure  may  not  cause  excessive  downtime  but  may  make  the 
equipment  unsafe  or  highly  failure -prone  during  operational ; 
use.  It  Is  thus  Important  to  recognise  that  the  passing  of  the 
demonstration  test  does  not  necessarily  mean  that  major  or  criti¬ 
cal  Improvements  cannot  or  should  not  be  made. 

Of  course,  if  the  equipment  passes  the  test,  the  contractor 
may  not  be  obliged  to  act  on  recommendations  resulting  from  the 
demonstration,  but  may  do  so  for  changes  that  can  be  easily  in¬ 
corporated.  If  the  recommended  changes  will  yield  a  significant 
improvement,  contract  renegotiation  may  be  called  for.  In  any 
event,  for  additional  procurements,  such  as  full-scale  production, 
the  customer  should  act  positively  on  the  information  provided  by 
the  results  of  the  demonstration-test  monitoring. 


"Ho  special  support  equipment  was  used  during  the  demon¬ 
stration  to  Isolate  the  inserted  faults.  A  module  puller 
was  used,  however,  to  remove  the  modules.  This  item  is 
not  required,  but  assists  in  module  removal  and  minimizes 
possible  module  damage.  It  is  recommended  that  a  module 
puller  be  procured  and  made  available  to  the  customer." 

"It  is  recommended  that  future  procurements  address  a  re¬ 
quirement  for  greater  ease  in  the  removal  and  replacement 
of  terminal  assemblies  than  now  exists.  Assembly  guide 
rails  and  guide  pins,  securing  devices  requiring  no  special 
tools,  and  use  of  receptacles  and  plugB  in  lieu  of  terminal 
boards  and  taper  pins,  where  appropriate,  are  cases  in 
point.  A  form  factor  permitting  convenient  removal  of 
assemblies  from  the  front  of  the  mounting  rackB  (without 
disturbing  other  assemblies)  is  an  Important  feature  for 
field  installations." 

"The  technicians  relied  almost  exclusively  on  the  handbook 
signal-flow  diagrams  for  fault  isolation  during  the  demon¬ 
stration,  and  they  commented  that  the  diagrams  were  superior 
to  conventional  schematic  diagrams .  On  the  basis  of  their 
effectiveness  in  the  demonstration,  it  is  recommended  that 
signal-flow  diagrams  be  incorporated  extensively  in  all  or¬ 
ganizational-level  maintenance  publications." 

4.  "Individual  maintenance  moc'ules  of  the  AN/3CYZ  are  suscepti¬ 
ble  to  damage  when  removed  from  the  equipment.  It  is 
recommended  that  suitable  containers  be  developed  to  pro¬ 
tect  modules  in  transit  to  and  from  repair  facilities." 

5.  "Replacement  of  the  Tube/Shield  Assembly  (AB123)  is  diffi¬ 
cult  and  time-consuming.  .During  the  demonstration,  re¬ 
placement  time  exceeded  45  minutes  for  two  technicians 
working  together. 

"It  is  recommended  that  captive  hardware  be  used  to  attach 
the  faceplate  and  the  Tube  Shield  Assembly  and  that  the 
yoke  and  ground  wires  be  connected  to  chassis  wiring  by  a 
bayonet-type  connector." 

"Connectors  C3,  C5,  and  G 6  are  all  the  same  size,  are  lo¬ 
cated  side-b„-side,  and  have  identical  external-plug  hous¬ 
ing  keying.  However,  the  internal  pins  of  each  connector 
are  arranged  in  patterns  different  from  other  connectors 


(continued) 


TABLE  Till  (continued) 


so  that  only  the  correct  plug  will  fit  tbs  connector.  «. 
problem  arises  In  that  the  plug  housings  will  fit  the  key¬ 
ing  on  any  of  the  four  connectors,  and  only  the  connector- 
pin  pattern  prevents  the  plug  mating.  If  sufficient  force 
Is  applied,  pins  can  be  damaged. 

"It  is  recommended  that  the  keying  on  all  four  of  these 
plugs  and  connectors  be  changed  so  that  each  pair  Is  unique^ 
preventing  the  possibility  of  pin  damage. " 

"The  module  guides  in  the  AN/FJK  equipment  used  in  the 
maintainability  demonstration  (serial  number  1)  Mere  very 
weak.  One  guide  was  broken  off  and  allowed  a  module  to  be 
inserted  skewed.  The  module  was  Improperly  seated,  which 
resulted  in  a  diode  failure.  Other  module  guides  were 
loose  and  allowed  the  modules  to  be  Improperly  seated  in 
their  plug-in  connectors.  The  contractor  has  noted  this 
design  deficiency,  and  has  Incorporated  an  Improved  module 
guide  that  is  much  stronger  than  the  guides  used  in  the 
serial  number  1  equipment.  The  contractor  stated  that  all 
subsequent  production  units  Will  contain  the  new  Improved 
module  guides." 

"The  front-access  modules  (Ml,  M2,  MS)  are  retained  in 
place  by  the  hinged  face  plate  of  the  indicator.  Techni¬ 
cians  on  two  occasions  during  the  demonstration  failed  to 
fully  Insert  tne  Selection/Deflection  Amplifier  (MS)  result 
ing  In  Improper  equipment  operation.  In  all  cases,  closing 
of  the  hinged  front  panel  was  difficult  because  of  Inter¬ 
ference  between  the  Tube/Shield  Assembly  (Ml)  and  the  bezel 
gasket  assembly. 


It  is  recommended  that  front-access  modules  be  retained 
by  fasteners  (preferably  quick- release  captive  type)  to 
ensure  full  insertion  cf  modules  in  their  receptacles,  and 
that  the  bezel/module  interference  be  corrected." 

"Lamp  Driver  Assembly  (L6)  circuit-card  replacement  is  ex¬ 
tremely  difficult.  The  technician  must  lean  over  the  ex¬ 
tended  Display  Tube  Subassembly  and,  at  the  limit  of  his 
reach,  release  two  quarter-turn  fasteners  with  a  screw¬ 
driver.  He  must  then  slide  the  assembly  toward  himself  - 
and  rotate  it  approximately  135  degrees  on  its  cables  for 
access  to  the  circuit  cards.  The  circuit  cards  require 
excessive  force  for  removal  and  insertion.  During  the 
demonstration,  a  technician  broke  one  the  cable  wires 
while  replacing  a  circuit  card  even  though  he  exercised 
great  care. 

"It  is  recommended  that  the  Lamp  Driver  Assembly  be  repack¬ 
aged  for  satisfactory  in-place  maintenance," 


SECTION  IV 


THE  STATISTICAL  BASIS  OP  MA  IOTA  D^HJTLY- DEMONSTRATION 

4.1  GENERAL 

A  maintainability-demonstration  test  provides  the  Informa¬ 
tion  necessary  for  accepting  or  rejecting  the  product  with  re¬ 
spect  to  conformance  to  stated  maintainability  goals.  The 
demonstration  effort,  therefore#  provides  the  input  to  a  deci¬ 
sion  process.  Since  this  input  Is  statistical  in  nature  — 
that  is,  the  results  of  the  test  are  not  constant,  but  are  sub¬ 
ject  to  statistical  (random)  fluctuation  —  it  is  necesorry  to 
consider  how  such  fluctuation  can  result  in  incorrect  decisions. 

In  practice,  an  accept  or  reject  decision  may  not  be  the 
only  alternative  (e.g.,  the  decision  may  be  to  "accept"  after 
a  specified  design  change  is  made).  However,  it  is  easiest  tc 
look  at  the  maintainability-demonstration  effort  from  the  ac¬ 
cept/reject  viewpoint  to  avoid  having  to  consider  all  possible 
types  of  decisions  that  can  be  made. 

4.2  DECISION  ERRORS 

If  the  discussion  is  restricted  to  accept/reject  decisions, 
two  basic  types  of  errors  can  be  made: 

Type  I  Error:  Reject  the  equipment  for  not  meeting  its 
maintainability  requirement  when.  In  fact, 
it  has. 

Type  II  Error:  Accept  the  equipment  as  meeting  Its  main¬ 
tainability  requirement  when,  in  fact,  It 
has  not. 

The  Type  I  error  is  detrimental  primarily  to  the  producer 
since  his  acceptable  equipment  is  being  rejected.  The  consumer 
also  wants  to  minimize  this  error  since  he  often  has  immediate 
need  for  the  equipment  and  would  not  want  to  experience  the  tan- 
necessary  delay  caused  by  this  type  of  error.  In  the  terminology 
of  acceptance  testing,  this  error  is  often  denoted  by  alpha  (a) 
and  is  called  the  producer's  risk. 

The  Type  II  error  is  detrimental  primarily  to  the  consumer 
since  he  Is  accepting  equipment  that  is  below  standard.  Unless 
there  is  a  contractual  requirement  for  the  producer  to  maintain 
acceptable  operational  performance,  he  has  no  direct  interest 
In  the  Type  II  error.  This  type  of  error  is  called  the  con¬ 
sumer's  risk  and  is  usually  denoted  by  beta  ( ^) . 
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It  Is  emphasized  that  for  any  type  of  dec.i  sion  procedure 
in  which  acceptable  and  unacceptable  levels  of  maintainability 
can  be  defined,  the  Type  I  and  Type  II  errors  are  always  in 
force.  The  statistical  theory  of  acceptance  testing  allows  for 
controlling  such  errors  by  controlling  the  sample  size,  select¬ 
ing  appropriate  test  statistics,  and  invoking  an  applicable  de¬ 
cision  criterion. 

4.3  THE  TEST  HYPOTHESIS 

Because  of  the  statistical  nature  of  demonstration  testing, 
the  basis  for  decision  is  developed  from  the  statistical  theory 
of  hypothesis  testing.  In  maintainability  testing,  the  hypothe¬ 
sis  under  test  --  called  the  null  hypothesis  and  denoted  by 
Hq  —  is  usually  that  the  submitted  product  conforms  to  the 

maintainability  requirement.  An  alternative  hypothesis,  de¬ 
noted  by  H-,,  is  also  specified  (or  implied);  it  statec  that  the 
product  is^at  an  undesirable  maintainability  level.  Rejection 
of  the  null  hypothesis  is  equivalent  to  acceptance  of  the  alter¬ 
native  hypothesis. 

For  example,  assume  that  the  parameter  of  Interest  is  mean 
corrective-maintenance  time,  denoted  by  TTct .  Then  two  basic 


forms  of  test  hypotheses  are 

as  follows: 

TEST  I 

TEST  II 

Ho:  =  30  mln 

Ho:  Mct  =  30  min 

H^:  TTct  =  60  min 

HI:  >  30  min 

For  Test  I,  Hq  and  H-^  specify  unique  values  for  and 

with  respect  to  this  characteristic  are  called  simple.  Strictly 
speaking,  a  simple  hypothesis  is  one  that  completely  specifies 
the  distribution  of  the  random  variable.  If  maintenance  time 

is  exponential  —  i.  e.,  f(x)  =  “  e-X/9 ~e  is  uniquely  deter¬ 
mined  by  a  mean  specification  such  as  those  of  Test  I.  For  a 
two-parameter  distribution  such  as  the  normal  or  lognormal, 
neither  HQ  nor  H^  of  Tests  I  and  II  uniquely  determines  the 

distribution  For  Test  II,  H1  does  not  specify  a  unique  value 
and  is,  therefore,  called  composite  with  respect  to  Mct 

The  relationship  between  the  test  hypotheses  and  the  de¬ 
cision  errors  is  now  evident.  For  both  ^ats  I  and  II  above, 

H  represents  the  mere  desirable  maintainability  level,  and 
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the  rejection  of  HQ  when  it  is  true  would  represent  the  proau- 
er's  risk  a.  for  Test  I  represents  an  undesirable  level  of 
maintainability,  and  acceptance  of  an  equipment  (equivalent  to 
accepting  H  )  when  H-  is  true  is  a  risk  to  the  consumer.  Test 
II,  however,  does  not  specify  a  unique  value  for  therefore, 
the  consumer's  risk  cannot  be  evaluated. 

If  the  Test  I  hypotheses  were  fco  be  invoked  and  the 
necessary  distribution  assumptions  on  maintenance  time  were 
validated,  the  assignment  of  a  and  0  risks  corresponding,  re¬ 
spectively,  to  rejecting  HQ  when  it  is  true  and  accepting  H.  if 

H,  is  true  would  generally  be  sufficient  for  determining  the 

sample  size,  n,  test  statistic,  and  decision  criterion  to  meet 
these  risk  levels. 

If  the  Test  II  hypotheses  are  to  be  invoked,  the  general 
procedure  is  to  use  as  large  a  value  of  n  as  possible,  since 
this  will  minimize  the  a  risk  as  well  as  minimizing  the  chance 
of  accepting  Hq  when  it  is  false.  This  conclusion  is  simply  a 

result  of  the  fact  that  the  variance  of  an  estlu*ite  generally 
decreases  ae  the  sample  size  increases,  and,  thus  the  larger 
n  Is,  the  more  precision  in  a  sample  statistic  snl  the  less 
risk  of  an  Incorrect  decision. 

Because  of  the  importance  of  the  test  hypotheses  and  asso¬ 
ciated  risks,  a  separate  section  (Section  V,  The  Maintaina¬ 
bility  Demonstration  Specification),  is  presented  to  provide 
guidelines  for  parameter  and  risk  specification. 

4.3.1  Specification  in  Terms  of  Confidence  Level 

Test  .requirements  are  sometimes  specified  in  terms  of  con¬ 
fidence  levels.  Such  a  specification,  however,  is  subject  to 
serious  misinterpretation,  as  the  following  example  illustrates. 
Assume  that  the  specification  states;  " .  .  .a  sample  shall  be 
tested  to  determine  with  90-percent  confidence  that  the  equip¬ 
ment  conforms  to  the  requirement  of  a  1/2-hour  mean  time  to 
perform  corrective  maintenance.  .  Two  reasonable  test  cri¬ 
teria  are: 

Test;  I:  Compute  the  90-percent  lower  confidence  limit,  Uj  . 

Since  there  is  90-percent  confidence  that  the  true 
mean  time  to  repair  is  greater  than  if  jijj>l/2, 
reject  the  equipment;  otherwise,  accept  it. 


Compute  the  °c~perc«nt  upper  confidence  limit, 

Since  there  if  90-percent  confidence  that  the  true 
mean  time  to  repair  is  leas  than  then  if  1 

accept  the  equipment;  otherwiee,  reject  it. 


Teat  I  ie  equivalent  to  one  in  which  the  producer's  risk  is 
10  percent  at  a  true  mean  of  1/2  hour.  Test  II  is  equivalent  to 
one  in  which  the  consumer's  risk  is  10  percent  at  a  true  mean  of 
1/2 -hour.  The  difference  between  the  two  tests  is  apparent:  The 
former  requires  that  equipment  with  a  mean  corrective -maintenance 
time  of  1/2 -hour  be  accepted  90  percent  of  the  time;  the  latter 
requires  acceptance  only  10  percent  of  the  time  if  the  equipment 
M  ,  is  1/2 -hour. 


Most  specifications  of  this  form  are  designed  to  represent 
a  Test  II  criterion.  This  criterion  makes  no  provision  for  the 
producer's  risk  at  a  highly  acceptable  maintainability  level. 
Many  plans  will  meet  the  criteria  of  Test  II.  Generally,  the 
lower  the  sample  size  the  higher  the  producer's  risk  for  a 
fixed  confidence  and  maintainability  level. 


In  any  case,  if  a  test  specification  is  to  be  made  in  the 
form  cf  a  confidence  interval,  it  is  imperative  that  it  be  made 
clear  whether  the  maintainability  liuraeric  represents  an  accept¬ 
able  or  unacceptable  maintainability  level. 

4.4  RELATIONSHIP  OP  RISKS  TO  SAMPLE  SIZE 


For  most  tests,  the  magnitude  of  a  and  p  and  the  number 
of  test  observations,  n,  are  Interrelated  in  such  a  manner 
that  specifying  any  two  of  the  quantities  determines  the  third. 
In  the  past,  for  nonsequential  tests,  a  and  n  were  usually 
specified  and  a  test  was  chosen  to  minimize  the  £  error.  For 
acceptance  teeting,  the  trend  now  is  to  specify  (3  instead  of  a. 
If  it  is  important  that  both  a  and  p  be  specified,  the  sample 
size  is  no  longer  at  the  discretion  of  the  experimenter,  as 
shown  for  many  of  the  fixed- sample- size  plans  presented  in  Sec¬ 
tion  VII.  In  sequential  sampling,  a  end  P  must  be  specified  in 
advance,  and  the  sample  size  is  a  random  variable  since  its 
value  is  not  predetermined  but  will  vary  over  successive  tests. 

4.5  THE  OPERATING- CHARACTERISTIC  (C.C)  CURVE 

By  specifying  two  of  tne  three  quantities  n,  a,  ana  p,  the 
accept-reject  criterion  of  the  acceptance  test  is  uniquely  de¬ 
termined  for  a  given  family  of  tests  (e.g.,  a  fixed  sample  test 
under  the  lognormal  assumption  with  known  variance).  It  is  then 
possible  to  generate  the  O.C,  curve  of  the  test  plan.  This 
curve  shows  the  probability  of  acceptance  over  all  possible  in¬ 
coming  maintainability  levels.  Two  points  on  the  O.C.  curve 
are  already  determined  --  the  a  and  p  points  with  their  corres¬ 
ponding  maintainability  levels,  which  are  given  by  H  and  H, , 
respectively.  1 


For  example,  assume  that  the  specification  Is  In  terms  of 
Sot,  the  mean  corrective  maintenance  time,  and  that  HQ  is  Bot  -  « 
minutes  and  is  1 *  40  minutes.  The  a  risk  is  0.10,  and  the 
6  risk  is  0.20.  The  general  shape  of  the  O.C.  curve  will  then 
be  as  shown  in  Figure  1. 

The  probability  of  acceptance  can  be  interpreted  as  the 
long-run  proportion  of  equipment  or  lots  that  will  be  accepted. 

If,  for  example,  the  O.C  curve  shows  that  an  of  25  minutes 
will  be  accepted  with  a  probability  of  0.65,  then  in  the  long 
run  65  percent  of  all  incoming  products  with  a  25-minutes  Mct 


will  be  accepted. 


For  the  acceptance  test:  HQ :  Mct  *  15  minutes 
Mct  =  40  minutes,  a  =  0.10,  p  =  0.20. 


FIGURE  1 

TYPICAL  O.C.  CURVE 
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4.6  TYPES  OP  MAINTAINABILITY  EECISION  TESTS 

A  decision  test  may  be  classified  in  many  different  ways. 
Some  of  the  more  Important  categorizations  of  maintainability 
tests  are  discussed  below. 

4.6.1  Type  of  Measurement 

Measurement  type  is  usually  categorized  in  terms  of  attri¬ 
butes  or  classification  data  and  variable  or  measurement  data. 

In  maintainability  testing,  the  usual  attribute-type  test 
is  one  in  which  a  success/failure  determination  is  made  on  each 
sample  observation  according  to  some  pre-established  criterion. 
Thus  if  the  maintainability  requirement  is  related  to  maximum 
duration  of  repair  times,  a  corresponding  attribute  measurement 
is  that  a  particular  maintenance-task  observation  did  or  did  not 
exceed  a  specified  maximum  time.  The  actual  time  spent  on  the 
task  is  not  directly  used  in  the  decision  criterion. 

A  variable  measurement,  on  the  other  hand,  doeB  employ  ac¬ 
tual  measurement  of  a  random  variable  that  is  continuously  dis¬ 
tributed.  For  maintainability  tests,  such  random  variables  are 
usually  maintenance  times; 

For  cases  in  which  either  type  cf  measurement  may  be  em¬ 
ployed,  such  factors  as  type  of  informat  on  provided,  degree 
of  protection  afforded,  amount  and  cost  of  inspection,  and  ease 
of  administration  should  be  considered.  Table  IX  summarizes  the 
advantages  and  disadvantages  of  each  type  of  measurement  with 
respect  to  these  considerations. 

4.6.2  Type  of  Maintenance-Task- Sample  Selection 

Testing  a  system's  maintainability  requires  sampling  from 
the  various  possible  types  of  maintenance  tasks  that  comprise 
the  hypothetical  total  population  of  maintenance  tasks.  The 
major  alternatives  to  consider  are  whether  induced  (simulated) 
failures  or  naturally  occurring  failures  are  to  be  considered 
end,  if  the  former  method  is  chosen,  whether  simple  random  sam¬ 
pling  Is  appropriate.  Section  Vi  covers  these  alternatives  in 
detail. 

4.6.3  Single,  Multiple,  and  Sequential  Sampling 

Single,  multiple,  and  sequential  sampling  plans  car  general 
ly  be  devised  such  that  each  affords  the  same  degree  of  protec¬ 
tion  or  has  nearly  identical  O.C.  curves.  For  convenience  In 
the  discussion*  that  follows,  an  attributes  test  is  assumed  -where 
by  a  sampled  maintenance  task  is  categorized  to  be  either  a 
success  or  a  failure  according  to  whether  the  maintenance  time 
is  less  than  or  greater  than  some  specified  number.  For  ex¬ 
ample,  such  o  criterion  might  be  used  to  test  whether  80  percent 
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TABLE  IX 


COMPARISON  BETWEEN  ATTRIBUTES  TEST 
AND  VARIABLES  TEST 


Factor 

Attributes  Test 

Variables  Test 

Type  of 

Information 

Yielded 

Number  or  percent  of 
sample  that  meets 
some  specified 
characteristic 

Observed  distribu¬ 
tion  of  some  quan¬ 
titative  output 

Type  of  Main¬ 
tainability 
Goal 

Meuian  or  percentile 
most  commonly  used 

Mean,  median, 
percentile,  and 
variance  are  most 
common 

Sample-Size 

Requirements 

Higher  than  variables 
test  for  correspond¬ 
ing  plan 

Lower  than  for 
attributes  test 
for  corresponding 
plan 

Ease  of 
Application 

Data  recording  and 
analysis  relatively 
simple 

More  clerical  and 
analysis  costs  than 
for  attribute  plans 

Statistical 

Consider¬ 

ations 

Applies*  to  both 
parametric  and 
nonparametrlc  tests 

_ 

Requires  an  assump¬ 
tion  on  the  under¬ 
lying  distribution 
unless  large  sample 
properties  are 
assumed 

of  all  maintenance  actions  take  less  than  20  minuteB  —  *  bi¬ 
nomial-type  test  in  which  the  hypothesis  Is  that  20  minutes 
Is  the  80th  percentile. 

In  single  sampling,  one  sample  of  n  items  is  tested.  Ac¬ 
cept  or  reject  decisions  are  made  on  the  basi3  of  the  results 
by  comparing  the  number  of  observed  unacceptable  malntenare 
actions  (i.e.i  one  that  takes  longer  than  20  minutes)  wit., 
predetermined  acceptance  number,  c.  In  multiple  sampling, 
more  than  one  sample  may  be  necessary  before  a  decision  is 


ill 


reached,  but  the  maximum  number  of  samples  and  thus  the  maxi¬ 
mum  number  of  items  to  be  tested  Is  known.  An  example  is  a 
double  sample  plan  with  the  following  test  criteria: 

(1st  sample  size)  =  20 

(accept  number  for  first  sample)  «  3 


ng  (2nd  sample  size)  *  40 

cg  (accept  number  for  both  samples)  «  7 

A  first  sample  of  20  items  is  taken.  If  3  or  fewer  unac¬ 
ceptable  maintenance  actions  are  found,  an  accept  decision  is 
made.  If  8  or  more 'unacceptable  maintenance  actions  are  found, 
rejection  takes  place.  If  4  to  7  unacceptable  maintenance  ac¬ 
tions  are  found  on  the  first  sample,  a  second  sample  of  40  items 
is  taken,  and  an  accept  decision  is  made  if  the  total  number  of 
unacceptable  maintenance  actions  is  7  or  fewer. 

Sequential  sampling  is  an  extension  of  multiple  sampling 
in  that  decision  to  accept,  reject,  or  sample  further  can  be 
made  after  each  individual  item  (or  possibly  group  of  items) 
is  tested.  For  a  standard  sequential  plan,  no  maximum  number 
of  sample  items  is  specified,  although  the  probability  of  very 
large  samples  is  usually  quite  small.  The  decision  criteria 
of  a  sequential  sampling  plan  can  be  presented  graphically. 

Figure  2  illustrates  a  sequential  test  based  on  a  binomial  dis¬ 
tribution  where  the  number  of  unacceptable  maintenance  actions 
is  the  decision  statistic. 

As  sampling  progresses,  the  number  of  unacceptable  main¬ 
tenance  actions  is  plotted  against  the  number  of  items  t'  ed. 
Testing  is  continued  until  the  plotted  step  function  cros.-es 
one  of  the  two  decision  lines.  Since  the  step  function  may  re¬ 
main  in  the  continuous  testing  region  for  a  long  period,  especi¬ 
ally  for  borderline  lots,  truncation  or  stopping  rules  can  be 
specified  so  that  the  effect  on  the  a  and  (3  errors  is  negligible. 

Multiple  sampling  generally  requires  less  testing  than 
single  sampling,  and  sequential  sampling  requires  less  testing 
than  multiple,  sampling  --  because  lots  with  very  good  or  very 
poor  quality  will  exhibit  such  characteristics  early  in  the 
testing  and  decisions  can  be  made  before  multiple  samples  or 
further  samples  in  a  sequential  test  are  required.  Since  the 
first  sample  of  a  multiple  sampling  plan  is  always  smaller  than 
a  single  sample  and  since  decisions  on  sequential  tests  can  La 
made  after  the  results  are  obtained  for  each  test  item,  such 
savings  in  sample  size  can  be  extensive  It  is  emphasized  that 
the  exact  sample  size  of  multiple  or  sequential  sampling  plans 
is  not  predetermined  but  is  a  function  of  the  quality  of  the 
submitted  product .  The  average  sample  for  various  levels  of 
incoming  quality  can  be  computed,  and  the  results  can  be  plotted 
to  yield  an  average  sample  number  (ASN)  curve. 
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FIGURE  2 

GRAPHICAL  REPRESENTATION  OF 
SEQUENTIAL  ACCEPTANCE  TEST 


An  example  of  these  curves  is  shown  in  Figure  3  for  a  main¬ 
tainability-demonstration  test  in  which  the  null  hypothesis  is 
that  20  minutes  is  the  95th  percentile  versus  the  alternative 
hypothesis  that  20  minutes  is  the  80th  percentile.  The  a  risk  is 
0.10^  and  the  (3  risk  is  0.05. The  single  sampling  plan  is  N  «  50, 

C  =  4.  Curves  for  equivalent  double  and  sequential  plans  are 
shown  in  the  figure, 

-  ^ 

Table  X  is  a  summary  comparison  of  some  characteristics  of 
single,  multiple,  and  sequential  sampling  plans. 

4.6.4  Parametric  and  Nonparpmetric  Tests 


A  parametric  test  is  one  in  which  the  underlying  proba¬ 
bility  law  of  the  random  variable  is  assumed  to  take  a  specific 
form.  In.  parametric  maintainability  tests,  for  example,  it  is 
often  assumed  that  maintenance  time  1b  a  random  variable  that 


20  30  40 

Number  of  Tests 
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Average  Sample  Size  Before  Decision 


Twenty  Minutes  '  m 


FIGURE  3 

AVERAGE  AMOUNT  OF  INSPECTION  UNDER  SINGLE,  DOUBLE, 
SEQUENTIAL  SAMPLING  ~  BINOMIAL  TEST  FOR  PERCENTILE 
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can  be  adequately  described  by  a  log-normal  distribution.  Non- 
parametnc  or  distribution-free  tests  are  those  in  which  no  as¬ 
sumptions  about  the  underlying  probability  law  are  made.  It  la 
noted  that  nonparametric  tests  do  not  deal  with  magnitude  but 
with  attribute  characteristics  such  as  rank,  frequency,  and 
ordinal  position. 

Although  the  distribution  of  the  attribute  tested  must  be 
known,  it  can  be  inferred  without  knowing  the  population  dis¬ 
tribution  of  the  basic  random  variable  For  example,  the  number 
of  maintenance- task  times  (the  attrioute)  less  than  a  constant  is 
a  binomial-distributed  random  variable  under  some  genei’al  require¬ 
ments,  irrespective  of  the  distribution  of  task  times  (the  basic 
random  variable). 

Generally,  parametric  tests  are  more  efficient  than  non- 
parametric  tests  since,  for  a  given  amount  of  testing,  more  pre¬ 
cise  estimates  or  smaller  probabilities  of  incorrect  decisions 
will  result  than  for  nonparametric  tests.  The  limitations  on 
the  types  of  parameters  that  can  be  tested  constitute  a  disad¬ 
vantage  of  nonparametric  tests.  For  example,  nonparametric 
tests  of  central  tendency  apply  only  to  the  median,  while  the 
specl'.'i  cat  ions  may  be  in  terms  of  the  mean. 

It  is  emphasized  that  an  incorrect  assumption  of  the  under¬ 
lying  probability  distribution  In  a  parametric  test  can  lead  to 
an  O.C.  curve  that  differs  greatly  from  that  planned,  especially 
for  small  sample  sizes.  Also,  nonparametric  tests  are  generally 
easy  to  conduct  and  evaluate,  often  requiring  only  counting, 
adding,  subtracting,  or  ranking.  Because  of  these  two  features, 
nonparametric  tests  are  now  receiving  much  more  consideration 
than  in  the  past . 

4.6.5  Classical  and  Bayesian  Tests 

A  Bayesian  test  can  be  generally  defined  as  one  that  employs 
prior  information  in  the  decision  criterion  concerning  the  ran¬ 
dom  variable  of  interest.  The  discussion  up  to  this  point  has 
been  primarily  concerned  with  classical- type  tests.  It  is 
noted  thac  prior  information  is  used  in  developing  a  sampling 
procedure  for  stratified  sampling,  but  such  use  involves  test 
design  rather  than  the  decision  criterion. 

The  classical  test,  as  shown  earlier,  involves  a  decision 
criterion  based  on  prescribed  probabilities  of  acceptance  for 
specified  maintainability  levels.  In  a  Bayesian  test,  the  test 
results  are  combined  with  the  prior  information  to  yield  a  re¬ 
vised  (Bayesian)  estimate  of  the  actual  distribution  or  para¬ 
meter,  and  decisions  are  made  according  to  the  desirability  of 
this  estimate. 
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In  essence,  if  an  1*1  ,  of  one  hour  i3  highly  desirable,  the 
classical  test  is  designed  such  that  P  [accept|M  1  hr]  is 

high.  A  Bayesian  test  would  be  one  for  which  Pf$  *  1  hr  I  accept] 
is  high.  ct 

Classical  tests  as  defined  here  are  well  documented,  have 
been  shown  to  provide  the  necessary  protection  against  rejecting 
good  or  accepting  bad  product,  and  thus  have  been  accepted  as 
being  a  reasonable  approach  towards  assuring  product  quality. 

Bayesian  tests  are  relatively  new  and  their  application 
will  therefore  have  to  undergo  a  trial-and-error  and  learning 
process.  They  do  possess  two  distinct  advantages  over  classi¬ 
cal  tests:  (1)  they  provide  for  using  available  information 
and  therefore  have  the  inherent  capability  of  reducing  the 
test  time  required  before  a  decision  is  made;  and  (2)  they  can 
provide  assurance  on  the  distribution  of  outgoing  or  accepted 
product,  while  classical  tests  generally  provide  no  such  con¬ 
trol  per  se . 

The  major  objection  to  Bayesian  tests  has  been  the  strong 
dependence  on  a  prior  distribution  —  the  existence  of  which 
some  deny  and  others  claim  cannot  easily  be  obtained  so  as  to 
be  useful. 

The  gx’eat  Interest  in  Bayesian  statistics,  as  evidenced  by 
recent  research  in  statistical  theory  and  applications  to  main¬ 
tainability,  reliability,  and  associated  disciplines,  points  to 
greater  use  of  Bayesian  approaches  In  the  future. 
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SECTION  V 


MAINTAINABILITY-DEMONSTRATION-TEST  SPECIFICATION 
5.1  GENERAL 

A  maintainability-demonstration-test  specification  is 
defined  here  as  a  set  of  numerical  requirements  and  associated 
risk  levels  that  will  govern  the  design  and  decision  criteria  of 
the  test.  For  the  most  common  tests,  this  specification  involves 
decisions  regarding  the  following: 

•  Type  of  maintainability  index  to  be  specified 

•  Acceptable  and  unacceptable  values  of  the  index 

•  Associated  risk  levels 

For  example,  the  test  specification  might  be  as  follows: 

Hq:  Mean  corrective-mint  •'nance  man-hours  =  40  minutes 
H]_:  Mean  corrective-maintenance  man-hours  =  80  minutes 
a  =  0.20,  ?  =  0.10 

A  test  based  on  this  specification  must  be  designed  such 

that 


P  (reject  |  MR.^.  =  40  min)  =  0.20 

_ t 

P  (accept  I  MH^  =  80  min)  =  0.10 

The  following  are  seme  of  the  more  important  requisites  fop 
a  maintainability-demonstration-test  specification: 

•  T.ie  maintainability  index  should  represent  a  measure  that 
is  directly  influenced  by  equipment  design  so  that  the 
producer  can  plan  for  high  assurance  of  a  pass  decision, 
but  bears  the  responsibility  for  a  reject  decision. 

•  Relationships  (at  least  qualitative)  between  design 
parameters  and  the  maintainability  index  should  be  known 
so  that  design  evaluations  and  predictions  are  possible. 

•  The  maintainability  index  should  be  appropriate  for,  and 
measurable  in,  the  demonstration- test  environment. 

•  The  maintainability  index  should  be  related  to  higher- 
level  system-requirement  parameters,  and  numerical  values 
should  be  consistent  with  values  for  these  higher-level 
parameters . 
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•  Adequate  sampling  ari  statistical-evaluation  procedures 
should  be  available  for  demonstrating  conformance  to  the 
requirement . 

•  Specified  maintainability  index  and  risk  values  should  net 
lead  to  sample  sizes  that  would  exceed  available  test 
resources . 

Not  all  of  these  requisites  are  necessarily  consistent,  and 
often  they  cannot  all  be  adequately  satisfied.  A  requirement 
consistent  with  higher-level  goals  may  result  in  specified  values 
that  require  sample  sizes  larger  than  expected.  Tests  for  con¬ 
formance  to  certain  types  of  requirements  may  require  complex 
statistical  tests  that  may  not  be  desirable. 

It  is,  therefore,  inportant  that  the  demonstration-test 
specification  be  prepared  as  early  as  possible  so  that  its  im¬ 
plications  can  be  fully  evaluated.  This  will  then  allow  time  for 
a  trade-off  analysis  between  test  costs  arid  risks  of  incorrect 
decisions. 

•Further  details  on  the  three  major  factors  of  a  maintain¬ 
ability-demonstration  specification  are  discussed  in  the  remain¬ 
der  of  this  section. 

5.2  TYPE  OF  SPECIFIED  MAINTAINABILITY  INDEX 

There  are  many  different  types  of  indices  that  can  be  speci¬ 
fied  for  a  maintainability  demonstration.  Some  of  the  more  usual 
alternatives  for  three  major  factors  are  as  follows: 

Factor  Alternatives 

✓ 

Type  of  Maintenance  Action  Corrective  maintenance, 

preventive  maintenance, 
total  maintenance 

Type  of  Statistical  Measure  Mean,  median,  variance, 

percentile 

Type  of  Time  Measurement  Equipment  downtime,  man¬ 

hours,  man-hours  per 
operating  hour 

The  above  listing  represents  a  possible  36  alternatives* 
two  such  are  mean  corrective-maintenance  man-hours,  and  the  95th 
percentile  of  equipment  downtime  due  to  all  types  of  maintenance. 
In  addition,  there  may  oe  multiple  parameters  such  as  a  mean  and 
percentile  and  specification  of  higher-level  indices  that  include 
maintainability  such  as  availability  or  effectiveness. 
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The  choice  of  an  appropriate  form  of  the  maintainability 
index  can,  therefore,  be  a  difficult  one.  It  is  the  purpose  of 
this  section  to  provide  some  guidelines  for  selecting  an  index- 
appropriate  for  the  maintainability- demonstration  task. 

It  is  emphasised  that  one  of  the  basic  purposes  of  a  main¬ 
tainability  demonstration  is  to  provide  assurance  of  accepting 
equipment  with  satisfactory  maintainability  characteristics. 

This  assurance  cannot  be  guaranteed  unless  the  definition  of 
"satisfactory  maintainability  characteristics"  is  established. 

This  is  not  usually  an  easy  task.  For  example,  the  operational 
commander  would  prefer  to  have  his  critically  needed  equipment 
operationally  ready  at  all  times.  The  base  maintenance  commander 
would  prefer  equipment  that  does  not  tax  his  manpower  organization, 
and  maintenance  man-hours  may  be  of  more  i’portance  to  him.  Cost 
control  may,  perhaps,  best  be  achieved  with  an  index  of  mainte¬ 
nance  nan-iiours  per  operating  hour. 

One  approach  that  at  first  might  appear  reasonable  is  to 
specify  several  types  ~>f  indices,  such  as  mean  corrective-main¬ 
tenance  time,  mean  number  of  corrective-maintenance  man-hours, 
and  man-hours  per  flight  hour.  However,  these  indices  are  re¬ 
lated,  and  it  is  quite  difficult  to  develop  and  apply  a  valid 
test  for  all  three  indices.  Even  so,  the  fact  that  they  are  re¬ 
lated  is  helpful  since  the  relationships  can  be  used  to  specify 
a  value  for  one  type  of  index  with  fair  confidence  that  an  ac¬ 
cepted  equipment  based  on  a  test  of  this  index  will  be  satisfac¬ 
tory  with  respect  to  the  related  index.  Some  of  the  more  impor¬ 
tant  relationships  are  reviewed  in  the  following  subsections  for 
each  of  the  three  factors  cited  above. 

5.2.1  Type  of  Maintenance  Action 

A  corrective  maintenance  action  is  one  performed  to  restore 
an  item  to  satisfactory  condition.  A  preventive  maintenance 
action  is  one  performed  to  detect  incipient  failures  or  prevent 
future  failures.  These  are  quite  general  definitions.  For  ex¬ 
ample,  in  practice,  for  many  supposedly  corrective-maintenance 
actions,  no  trouble  is  found  and,  therefore,  no  true  corrective 
action  is  perforiiied. 

For  specification  purposes,  the  distinguishing  feature  be¬ 
tween  the  two  types  of  maintenance  is  that,  generally,  corrective 
maintenance  is  unscheduled,  while  the  preventive-maintenance 
schedule  can  often  be  controlled. 

Therefore,  with  respect  tb  an  operational  requirement  such 
as  availability,  corrective  maintenance  is  more  critical  ii  pre¬ 
ventive  maintenance  can  be  scheduled  so  as  not  to  conflict  with 
operational  demands.  This  may  not  always  be  true,  however.  If 
the  Air  Force  requires  overhaul  of  a  Jet  engine  after  600  operat¬ 
ing  hours,  and  if  no  spares  are  available  and  the  aircraft  is 
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subject  to  random  demand*  the  preventive  melntenance  action  of 
overruling  the  engine  Is  as  critical  as  a  corrective  maintenance 
action.  If  a  spare  engine  is  installed  duriig  the  overliaul*  the 
importance  of  preventive-maintenance  time  with  respect  to 
operational  needs  is  dimin toned. 

The  frequency  and  duration  of  preventive  maintenance  actions 
directly  affects  iraintenance-manpower  control.  Again,  since  such 
actions  can  often  be  scheduled,  the  specification  of  a  corrective- 
maintenance  parameter  may  be  more  important  than  specifying 
preventive  maintenance. 

The  choice  of  whether  separate  indices  or  combined  indices 
(total  downtime  or  man-hours)  of  maintainability  should  be  used 
depends  on  several  factors.  If  corrective  maintenance  it;  more 
important  than  preventive  maintenance,  separate  indices  and  sep¬ 
arate  tests  may  be  preferred.  If  downtime  due  to  any  cause  is 
critical,  a  total-downtime  index  may  bo  used. 

Prom  the  statistical  viewpoint,  separate  tests  are  preferred 
since  the  distributions  of  the  two  type,;  of  actions  might  be  dif¬ 
ferent  and  combining  both  types  would  result  in  a  mixture  of  two 
distributions,  which  hinders  development  of  an  appropriate  test. 

There  is  generally  a  positive  correlation  between  the  sta¬ 
tistics  for  preventive  maintenance  and  those  for  corrective- 
maintenance.  Many  of  the  tasks  are  identical  (e.g,,  the  final- 
checkout  routine),  and  the  factors  that  represent  good  or  poor 
maintenance  characteristics  will  generally  affect  both  types  of 
maintenance  action  in  the  same  manner.  For  example,  if  pocr 
accessibility  is  a  major  contributor  to  excessive  corrective- 
maintenance  time  on  an  equipment,  it  will  also  adversely 
influence  the  preventive  maintenance  action. 

5.2.2  Statistical  Measures 

The  mean  and  median  are  central- tendency  parameters,  tie 
variance  Is  a  measure  of  spread,  and  the  percentile  specifica¬ 
tion  provides  a  control  on  extremes.  Mathematically,  If  x 
represents  the  random  variable  of  interest  and  f (x)  Its  contin¬ 
uous  probability-density  function,  the  following  are  defining 
relationships:  m 

Mean:  E(x)  =  J  xf(x)dx 

M(x) 

Median:  M(x)  =  the  value  of  x  for  which  J  f(x)dx  =  \ 

00  -  oc 

=  f  f(x)dx 
M(x) 
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Variance:  V(x)  -  E  [x-E^x)]^  »  f  [x-E(x)]-  f(x)dx 

-  00 

(l-p)th  Percentile:  Xp  =  the  value  of  x  for  wnich 

00 

Jx  f(x)dx  =  p 


For  symmetrical  distributions,  the  mean  and  median  coincide. 
The  median  is  also  the  50th  percentile.  An  indirect  control  on 
the  variance  can  be  provided  by  a  two-parameter  specification 
such  as  the  mean  and  95th  percentile  or  the  median  and  90th  per¬ 
centile.  To  demonstrate  this,  the  normal  distribution  with  the 
following  density  is  considered: 


f(x) 


_L_  e  2c2 

■12  ',:a 


(x-p)2 


where  p  is  the  mean  and  cr2  is  the  variance.  The  (l-p)th  per¬ 
centile  of  the  normal  distribution  is  given  by 


=  p  +  Z 


a 


where  Zp  is"  a  standardized  normal  deviate  corresponding  to  the 
(l-p)th  percentile  of  a  normal  distribution  with  mean  0  and 
variance  1.  For  p  =  0.05,  Zp  =  1.645.  Thus  if  the  mean  and  the 
95th  percentile  were  specified  to  be  j.  and  3,  respectively,  the 
following  would  be  obtained: 

•  tA' 

p  =  1.0  — 


xo.o5  58  3.0 

Then,  from  the  definition  of  XqqC, 

X0.05  =  M-  +  zp° 
or  3.0  =  1.0  +  1.6450 
or  o  =  =  1.22 
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The  distribution  most  applicable  to  maintenance  times  has 
been  found  to  be  the  lognormal  distribution.1  The  density  func¬ 
tion  is 

- ~~2  Un  x-S)2 

f  (X)  =  e  ?*  ,  x  >  o 

-/27T  nx 


If  a  random  variable  has  a  lognormal  distribution  function,  its 
logarithm  is  normally  distributed  with  mean  6  and  variances2. 
The  following  relationships  pertain  to  the  lognormal: 


Mean: 


e(x)  =  e 


9  +  a 


2/ 


2 


Median: 


m(x)  =  e9 


2  2 

Variance:  V(x)  =  S^9  +  °  (0J  -  1) 

(1-p)^*1  percentile:  Xp  -  6^  +  percentile  of 

normal  (0, 1) distribution. J 

Because  of  these  relationships,  the  specification  of  any  two 
of  the  above  parameters  completely  defines  the  distribution 
(see  Table  XXXII,  Section  VII ).  The  fact  that  the  median  is  Inde¬ 
pendent  of  is  one  reason  why  this  parameter  is  often  associ¬ 
ated  with  the  lognormal  since  it  allows  the  application  of  rela¬ 
tively  simple  tests. 

A  detailed  study  of  the  relationships  of  lognormal  parame¬ 
ters  and  the  specification  of  such  parameters  is  presented  in  the 
Rome  Air  Development  Center  Technical  Report,  RADC-TR-67-403, 
Maintainability  Parameters  and  Their  Relationships,  J.  Klion, 
September  19&7. 

5,2.3  Type  of  Time  Measurement 

Equipment  downtime  is  the  time  measurement  most  related  to 
operational  requirements.  Man-hours  and  man-hours  per  operating 
hour  are  more  closely  associated  with  manpower  and  cost  control 
although  they  are,  of  course,  related  to  downtime. 

Average  man-hours  per  maintenance  action  can  be  estimated 
by  multiplying  the  average  number  of  maintenance  men  per  mainte¬ 
nance  action  by  the  average  downtime. 


*£lee  Appendix  A  for  a  detailed  discussion  of  the  lognormal 
distribution. 


54 


For  13  equipments  monitored  in  tnis  study,  estimates  of 
total  active-maintenance  man-hours  (excluding  ''no  trouble  found" 
actions)  based  on  this  relationship  yielded  good  results.  The 
estimated  ancl  observed  man-hours,  along  with  the  absolute  value 
of  the  relative  error,  are  shown  in  Table  XI.  The  average  of  the 
relative  errors  is  7.6  percent. 


TABLE  XI 

OBSERVED  AND  ESTIMATED  MAINTENANCE  TIME 
(Maintenance  Man-Minutes) 


Equipment 

Observed 

Estimated 

Absolute 

Relative 

Error* 

1  A04 

60.3 

63.0 

0.046 

ao8 

282.6 

322.8 

0.142 

A10 

17^.6 

162.4 

(0 

• 

0 

-»3 

O 

bo4 

118.8 

106,8 

0.101 

bo6 

61.5 

57.8 

0.061 

B09 

81.1 

78.5 

0.032 

BIO 

98.1 

94.6 

0.036 

B12 

71.2 

1  79.5 

o.ii3 

B13. 

20.3 

22.5 

0.110 

Bl4 

71.0 

74-. 0 

0.043 

DOi 

155.8 

153.2 

0.017 

D02 

77.9 

cj.3 

0.162 

D03 

101.5 

114.6 

0.129 

* (observed  -  Estimated! 

Observed 

For  conplete  systems  such  as  aircraft,  for  which  concurrent 
maintenance  actions  can  and  frequently  do  take  place,  the  rela¬ 
tionship  between  equipment  downtime  and  system  downtime  provides 
a  basis  for  allocating  an  overall  system-downtime  requirement. 

If  a  system  is  divided  into  n  equipments  such  that  (1)  concur¬ 
rent  maintenance  can  take  place  only  on  different  equipments  arid 
not  within  an  equipment,  and  (2)  the  probability  that  more  than 
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two  equipments  will  require  concurrent  maintenance  is  negligible, 
then  the  following  relationship  can  be  used  as  a  irodel  for  aver¬ 
age  system  downtime  based  on  equipment  downtimes: 

nj  n-1  rvj  n  n£ 

1  P0*lJ  +  i  /_  X  L  pU,kjl'mx 

J=1  1=1  j=l  1=1  1=1+1  k=l 

where 

ts  =  average  system  downtime 

nj  is  the  number  of  possible  tasks  for  the  equipment 

Pjj  is  the  probability  that  only  the  i^h  task  in  the 
equipment  is  performed  given  system  failure 
Plj  is  the  probability  that  the  ith  task  in  the 

equipment  and  the  k^*1  task  in  the  ,2th  equipment  are 
required  concurrently  given  system  iailure 

*t1  .  is  the  average  time  for  completing  the  ith  task  in  the 
j1'"  equipment 

Three  models  for  estimating  system  man-hours  from  downtime 
estimates  are  shown  below.  The  following  notation  is  used: 

MHg  =  System  man-hours 

i%  =  Average  number  of  men  per  system  naintenance  action 

mg(r)  =  Average  number  of  men  per  system  maintenance  action 
when  r  failures  are  involved 

m^  =  Average  nuntoer  of  men  involved  when  the  ith  mainte¬ 
nance  task  is  required  for  the  equipment 

"Es  =  Average  system  downtime 

t3(r)  =  Average  system  downtime  when  r  failures  occur 

'■  LU 

Tjj  -  Average  downtime  of  the  j  equipment  when  the  i 
J  maintenance  task  Is  to  be  performed 

pr  =  Probability  that  r  maintenance  tasks  are  required 
given  system  failure 
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Pj_j  =  Probability  that  only  the  ich  maintenance  task  in 
the  equipment  is  required  for  a  system  failure 

Pi.1,k£=  Probability  thaf  the  maintenance  task  for  the 
jtn  equipment  and  kt!ri  maintenance  task  for  the  4th 
equipment  are  required  for  a  system  failure 

The  models  are  as  follows: 

A.  Model  based  on  average  system  downtime: 

E(MHs)  =  msts 

B.  Model  breed  on  average  system  downtime  as  a  function  of 
number  cf  equipment  failures: 

E(MH3)  =  £  Pr  ms(r)ts(r) 

r 

C.  Model  based  on  average  equipment  downtime  assuming  no 
more  than  two  concurrent  tasks: 

e(mhs)  =  y  y  Pij^tij  +v  y  y  y  pi^u  +  %****] 

L—i  l—i  <— j  l—i  !—i  l—! 

J  i  J  i  i  k 

Models  A  and  B  are  both  based  on  system-downtime  statistics. 
The  former  requires  only  an  overall  estimate  of  system  downtime 
and  manning,  while  the  latter  requires  estimates  of  downtime  and 
manning  as  a  function  of  the  nuntoer  of  concurrent  tasks  required. 
Since  manpower  is  generally  limited,  if  a  large  number  of  simul¬ 
taneous  failures  requiring  many  concurrent  maintenance  tasks 
occur,  system  downtime  may  be  partially  due  to  the  unavaila¬ 
bility  of  maintenance  men.  This  factor  is  accounted  for  directly 
in  Model  B  through  the  ts(r)  variable  but  only  indirectly  in 
Model  A. 

Model  C  uses  equipment  rather  than  system- downtime  values 
directly.  While  these  may  be  more  readily  available,  no  account¬ 
ing  is  made  for  total  system  checkout  after  all  tasks  are  com¬ 
pleted,  although  an  appropriate  "K"  factor  can  easily  be  incor¬ 
porated.  While  the  equation  shown  limits  the  number  of  concur¬ 
rent  tasks  to  two,  it  is  apparent  that  an  extension  can  be  made 
for  more  than  two  tasks. 
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A  me  .el  for  the  expected  value  of  man-hours  per  opei&ting 
hour  (MH/OH),  showing  its  relationship  to  average  downtime*  can 
be  developed  as  follows: 


E[ MH/OH I 


00 

^  P[j  failures  in  T  operating  hours ]E[MH 
j=0 


failures ] 


00 

1  7" 

T  '/  P[j  failures  in  T  operating  hours].j  E[MK|l  failure] 
j=0 


e[mh|1  failure] 


failures 


operating  hours. 


_  E[MH  failure]  g[nunjjer  Cf  failures  in  T  operating  hours] 

If  a  constant  equipment  failure  rate  is  assumed,  the  follow¬ 
ing  rate  is  obtained: 

E[MH/OH]  =  =  X  E  [  MH/ac  t  ion] 

Again,  by  using  the  relationship 

E[MH/action]  =  E [number  of  men/action] [average  downtime]  =  iT 
the  following  is  obtained: 

E[MK/0H]  =  MF 

This  general  expression  applies  at  either  the  equipment 
level  cr  the  system  level,  but  it  is  more  accurate  for  the  former 
because  the  sinplifying  equations  do  not  account  directly  for 
concurrent  maintenance,  it  is  emphasised  that  the  nan- hour  rate 
is  directly  influenced  by  reliability.  From  the  equation 

E[MH/OH]  =  7,  ml 

it  is  seen  that  the  man-hour  rate  changes  in  direct  proportion 
to  changes  in  the  failure  rate,  This  Is  not  true  for  average 
downtime  or  average  man-hours  per  maintenance  action,  since  the 
reliability  factor  for.  these  measures  influences  only  the  rela¬ 
tive  frequency  of  the  various  maintenance  tasks  given  that  a 
maintenance  action  is  required. 
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Since  a  demonstration  test  based  on  man- dorr  rate  is  a  test 
that  includes  both  reliability  and  maintainability  factors,  it 
does  not  truly  fall  in  the  category  of  a  maintainable ity-domon- 
station  test  in  which  the  maintainability-design  group  bears 
the  majo1  responsibility  for  success  in  meeting  goals.  This  is 
not  to  imply  that  a  test  based  on  man-hour  rate  is  not  useful; 
however,  it  is  important  to  recognize  the  influences  on  this 
type  of  measure  so  chat  appropriate  responsibilities  and  control 
can  be  estaolished. 

Because  of  uhe  direct  relationship  between  expected  man¬ 
hours  per  maintenance  action  and  man-hour  rate,  and  the  face  that 
control  of  tne  latter  is  as  much  a  responsibility  of  reliability 
design  as  of  maintainability  design,  man-hour  rate  as  a  maintain¬ 
ability  index  was  not  treated  in  as  much  depth  in  this  study  as 
the  downtime  and  man-hour  indices. 

The  reJationships  shown  above  that  relate  man-hours  and  man¬ 
hour  rates  to  equipment  downtimes  do  show  that  che  specification 
of  equipment  downtimes  does  permit  evaluation  of  man-hour  param¬ 
eters  at  either  the  system  or  equipment  level.  The  complexity 
of  the  relationships  for  system-level  requirements,  however,  d *  >s 
highlight  the  problems  that  can  occur  if  a  system  ma^-hour  re¬ 
quirement  is  to  be  tested  by  synthesizing  results  obtained  at  the 
equipment  level. 

*.2. 4  Guideline"  for  Index  Selection 

The  principal  objective  in  selecting  the  index  for  a  main¬ 
tainability-demonstration  test  should  be  to  seek  the  one  that  is 
most  consistent  with  the  mission  objectives  and  operational  con¬ 
straints.  Generally,  this  will  mean  that  equipment  downtime  is 
the  time  measurement  of  the  index  since  operational  effectiveness 
cannot  be  achieved  unless  downtime  is  controlled. 

If  the  need  for  an  equipment  is  not  critical,  and  manpower 
control  is  important,  a  man-hour  index  may  be  most  appropriate. 
Preventive-maintenance  man-hours  per  operating  hour  is  preferable 
to  downtime  due  to  preventive  maintenance  for  equipments  for 
which  such  maintenance  can  be  scheduled  without  fear  of  opera¬ 
tional  demand  during  the  maintenance  action. 

By  the  same  reasoning,  corrective  maintenance  is  more  crucial 
than  preventive  maintenance,  c-pecially  if  the  latter  can  be 
scheduled  to  take  place  during  known  periods  of  non-use.  For  con¬ 
tinuously  needed  equipment,  such  as  an  alert  radar,  total  main¬ 
tenance  time  is  of  prime  importance.  For  equipment  demanded  at 
random  times,  such  as  a  missile-defense  equipment,  the  approach 
might  be  to  use  separate  controls  for  corrective  maintenance  and 
preventive  maintenance.  The  choice  of  the  statistical  measure  to 
be  used  often  depends  on  the  mission  objective.  If  there  is  an 
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availability  requirement  for  the  system,  the  following  relation¬ 
ship  is  often  used  to  determine  reliability  and  maintainability 
requirements: 


Availability  - 

This  relationship  provides  a  basis  for  trade-off  between 
MTBF  and  MD7.  Several  such  trade-off  curves  are  shown  in  Sub¬ 
section  5.3»£  for  various  availability  requirements. 

When  this  availability  expression  is  appropriate,  a  mean 
value  should  be  used  for  the  maintainability  index.  However, 
there  may  be  an  availability  requirement,  for  which  a  maximum 
downtime  is  more  appropriate.  Such  a  requirement  would  apply  to 
critical  equipment  aboard  an  aircraft  where  the  aircraft  may 
have  to  be  available  for  a  new  mission  within  two  hours  after 
completing  a  mission.  In  this  case  a  requirement  of,  say,  0.95 
probability  of  completing  any  necessary  maintenance  within  100 
minutes  would  be  more  consistent  with  the  operational  objective 
tnan  a  mean-value  index.  This  is  discussed  in  greater  detail  in 
Subsection  5.3- 

Another  element  to  be  considered  in  the  choice  of  the  statis¬ 
tical  measure  is  the  underlying  distribution  of  maintenance  times. 
For  symmetrical  distributions,  the  mean  and  median  are  identical, 
and  the  choice  depends  solely  on  statistical  niceties.  For  the 
more  common  skewed  distributions  of  maintenance  time,  the  mean  is 
strongly  influenced  by  t^e  long  maintenance  times  but  the  median 
is  not.  When  either  can  be  used,  the  mean  generally  provides 
better  manpowox  cost  control,  is  derivable  from  a  higher- level 
specification,  and  has  more  desirable  statistical  properties 
(e.g.j  application  of  the  central-limit  theorem).  The  median  is 
applicable  to  distribution- flee  tests,  has  direct  operational 
meaning  In  the  sense  that  50  percent  of  all  maintenance  actions 
will  be  performed  within  the  median-time  period,  and  for  the  log¬ 
normal  distribution  is  dependent  on  only  one  parameter  as  con¬ 
trasted  with  two  for  the  mean. 

Ti:e  foregoing  discussion  has  provided  some  direction  on 
choosing  an  appropriate  maintainability  index  for  several  situ¬ 
ations.  To  provide  greatex  detail  and  permit,  more  definitive 
recommendations,  an  established^ matrix  for  selecting  maintain¬ 
ability  measures  is  used  here.*  In  this  procedure,  seven  con¬ 
ditions  that  should  influence  the  choice  of  a  measure  or  index 
are  listed,  and  recommended  indices  for  common  combinations  of 
such  conditions  are  given  In  the  matrix.  The  conditions  and 
matrix  oresented  here  are  modified  slightly  from  those  of  the 
Notebook  +o  make  them  applicable  to  demonstration. 


^THTIT'SYrTx  will  appear  in  the  "Maintainability  Engineering  De¬ 
sign  Notebook",  being  prepared  by  ARINC  Research  Corporation  for 
Rome  Air  Development  Center  under  C<  ntract,  F30602-68-C-0208. 


The  matrix  is  presented  in  Exhibit  2.  To  use  the  matrix, 
each  of  the  conditions  listed  at  the  top  of  the  exhibit  that 
apply  to  the  equipment  of  interest  should  be  checked.  The  ap¬ 
propriate  index  is  then  found  from  the  matrix  by  locating  the 
column  t'rxat  contains  an  x  for  each  condition  checked  above. 

For  example,  if  steady-state  availability  is  a  critical  parameter 
(Condition  1 j  and  maintenance  time  is  limited  by  environmental 
or  operational  circumstances  (Condition  5)j  the  recommended  index 
provides  a  control  on  both  the  mean  and  maximum  maintenance  time, 
and  there  is  an  option  for  including  preventive-naintenance  time 
depending  on  equipment  use  or  scheduling  and  criticality. 

The  set  of  conditions  listed  is  not  exhaustive,  but  it  is 
believed  to  include  the  most  important  ones. 

Several  of  the  major  considerations  that  led  to  the  develop¬ 
ment  of  the  matrix  are  as  follows: 

•  The  mean  is  directly  related  to  steady-state  availability 
and  is  therefore  the  index  of  choice  when  this  operational 
requirement  exists. 

•  If  the  distribution  of  maintenance  times  is  unknown,  the 
median  is  preferred  since  it  permits  distribution-free 
tests.  If  availability  is  critical,  however,  use  of  the 
central-limit  theorem  permits  a  mean  test  provided  the 
sample  size  is  large. 

•  For  the  lognormal  distribution,  the  median  Is  preferred 
to  the  mean  (assuming  that  Condition  2  applies  and  that  5 
and  6  do  not)  since  It  is  based  on  only  one  parameter, 
which  rakes  statistical  analysis  exact. 

*i  When  maintenance  time  is  limited  (Condition  5)*  the 
index  is  preferred. 

•  The  mean  Is  preferred  over  the  median  if  manpower  control 
is  also  required  because  the  mean  it>  more  directly  re¬ 
lated  to  man-hours.  However,  if  the  distribution  is 
unknown,  the  median  may  be  used  as  long  as  availability  is 
not  critical. 

Conplete  dependence  on  this  procedure  is  to  be  avoided.  Be¬ 
cause  of  the  wide  variety  of  equipments,  mission  objectives,  and 
environmental  and  operational  circumstances,  the  selection  matrix 
should  be  considered  a  guide  only.  Ultimately,  the  best  measure 
Is  determined  by  individual  system  circumstances  and  good 
judgment  c 
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Condition  Identification 
(Place  X  In  appropriaTe  boxes) 

Condition 

□  1  Steady-state  availability  Is  a  critical  parameter. 

□  2  Steady-state  availability  Is  not  a  critical  parameter. 

□  3  Maintenance- time  distribution  Is  unknown. 

□  4  Maintenance-time  distribution  Is  expected  to  be  lognormal. 

□  5  Environmental  or  operational  circumstances  limit  maintenance  time. 

□  6  Manpower  allocation  or  cost  Is  an  Important  factor. 


mean,  M  =  median,  M^^  =  maximum  maintenance  time  ,  MMH  =  maintenance  man-hours 

(percentile) 

corrective  maintenance,  pt  =  preventive  maintenance 

The  inclusion  of  preventive-maintenance  Indices  Is  optional  depending  on 
scheduling  and  criticality. 

A  contolned  total-maintenance- time  Index  can  be  used  Instead  of  separate 
Indices  for  corrective  and  preventive  maintenance. 

EXHIBIT  2 

PROCEDURE  FOR  MAINTAINABILITY -INDEX  SELECTION 


62 


5.3  SPECIFIED  VALUES  FOR  THE  MAINTAINABILITY  INDEX 
5.3.1  Three  Basic  Criteria 


As  discussed  In  Section  4.3,  the  usual  specification  of 
values  for  maintainability  demonstration  Involves  assignment  of 
two  values  for  the  Index  selected  — >  a  desirable  value  associ¬ 
ated  with  t.ie  null  hypothesis,  Ho,and  an  undesirable  ( some* tines 
called  marginally  acceptable)  value  associated  with  the  alterna¬ 
tive  hypothesis,  H^. 

In  assigning  such  values,  it  is  reasonable  first  to  con¬ 
sider  the  goal  or  Hq  value,  since  this  is  what  the  producer  and 
consumer  both  seek,  and  then  to  assign  the  Hj  value,  which  will 
be  a  function  of  the  desirable  value,,  minimum  operational  goals, 
and  other  factors  such  as  required  simple  sizes. 

There  are  three  basic  criteria  for  specifying  the  desirable 
values  of  the  selected  maintainability  index: 

(1)  The  specified  value  should  be  consistent  with  higher- 
systemrlevel  requirements. 

(2)  It  should  be  realistic. 

(3)  It  should  pertain  to  the  demonstration  environment. 

Unfortunately,  the  first  two  criteria  are  sometime^  in  con¬ 
flict  because  higher-level  requirements  may  be  unrealistic .  As  a 
minimum,  any  maintainability -index  value  derived  from  higher-level 
requirements  should  be  checked  for  reasonableness.  If  the  derived 
values  are  not  reasonable  (exceed  the  state  of  the  art),  the 
higher-level  goal  Is  suspect.  If  this  goal  cannot  be  compromised, 
however,  the  producer  can  be  given  some  reprieve  by  adjustment  of 
risk  values. 

On  the  other  hand,  if  a  derived  value  is  less  acceptable  than 
what  can  normally  be  expected,  the  values  should  be  readjusted  to 
state-of-the-art  levels. 

The  third  criterion  may  also  be  a  source  of  conflic*.  A 
higher-level  requirement  such  as  overall  system  effectiveness  may 
be  translated  into  capability  and  availability  requirements  by 
such  means  as  a  WSEIAC-type  analysis.  The  availability  must  then 
be  further  refined  to  reliability  and  maintainability  indices. 

The  operational  environment  to  which  the  maintainability  index 
applies,  however,  may  be  significantly  different  from  the  demon¬ 
stration  environment  under  which  the  equipment  or  system  is  to  he 
tested . 
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It  is,  naturally,  preferable  to  conduct  the  test  in  the 
actual  operational  environment,  but  t'l.ls  is  often  impossible. 

If  the  test  environment  does  differ  from  the  operational  en/iron- 
ment,  it  is  reasonable  to  adjust  a  maintainability  goal  based  on 
operational  need  to  refleot  the  differences  between  test  ana 
field  conditions. 

5.3.2  Specification  Based  on  Higher-System-- Level  Requirement  s 

There  are  many  types  of  system-level  requirements  that  are 
directly  or  indirectly  related  to  the  maintainability  character¬ 
istics  of  a  system.  The  most  important  of  these  system  operational- 
type  requirements  ie  availability,  which  in  the  most  general  sense 
is  a  measure  of  the  readiness  of  the  system  for  operational  use  on 
demand.  Two  of  the  more  importfcint  types  of  availability  measures 
are  as  follows: 

.  Point  Availability  -  The  probability  that  the  system  is 
available  for  operational,  use  at  a  random  point  in  time. 

.  Interval  Availability  -  The  probability  that  the  system 
will  be  available  for  operational  use  within  a  specified 
time  interval. 

Point  availability  is  generally  applicable  to  systems  whose 
mission  is  continuous,  such  as  an  alert  radar1.  For  these  types 
of  systems,  the  ratio  of  on  time  to  total  time  must  be  high,  and 
this  ratio  is  best  expressed  by  the  general  steady-state  avail¬ 
ability  expression 


where 


A  == 


MTBM 

MtEM  +  MDT 


MTBM  =  mean  time  between  maintenance 
MDT  =  mean  downtime 

When  preventive  or  noncorrective  maintenance  can  be 
scheduled  so  that  it  does  not  conflict  with  mission  objectives, 
the  following  expression  is  applicable: 


A 


MT3F 


where 


MTEF  «  mean  time  between  failures 
39TTR  »  mean  time  to  repair 
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Interval  availability  as  defined  above  Is  applicable  when 
the  system  is  required  to  perform  a  series  of  missions,  the  most 
common  example  of  such  a  system  being  an  aircraft.  For  such 
cases,  it  is  often  Important  to  control  the  probability  of  read¬ 
iness  after  completion  of  a  mission  by  an  interval- availability 
requirement . 

A  model  for  this  type  of  requirement  can  be  fairly  complex 
depending  on  the  system,  operational  conditions,  and  assumptions 
made.  A  relatively  simple  model  for  steady-state  interval  avail¬ 
ability  —  assuming  a  Markov  process  for  the  mission/service- 
repair  sequence,  constant  mission  time  T,  and  constant  allowable 
repair  time  t  —  is  presented  below. 

Let 

A(t)  represent  the  probability  that  the  system  is  available 
within  t  hours  after  scheduled  mission  completion 

R(T)  represent  the  system  reliability  for  a  mission  of  T 
hours 

S(t)  represent  the  probability  that  necessary  servicing 
(e.g.,  refueling  and  rearming  an  aircraft)  is  performed 
within  t  hours  after  a  successful  mission 

M( t )  represent  the  probability  that  servicing  and  any  neces¬ 
sary  repairs  can  be  accomplished  within  t  hours  after  initi¬ 
ation  of  maintenance  on  a  failed  system 

The  steady-state  interval  availability  Is  then  given  by  the 
following  equation  [a_bar  above  a  symbol  represents  the  comple¬ 
mentary  event,  e.g.,  R(t)  =  1  -  R(t)J: 

A(t)  =  A(t)R(T)S(t)  +  A(t)IT(T}M(t)  -r  A(t )M(T  +  t) 


The  first  term  on  the  right-hand  side  is  the  probability 
that  the  system  was  available  at  the  start  of  the  previous 
mission,  did  not  fail  in  T  hours  of  operation,  and  Is  serviced 
within  t  hours.  The  second  term  represents  the  probability  that 
the  system  was  available  at  the  start  of  the  previous  mission,, 
a  failure  occurred  during  that  mission,  and  repair  and  servicing 
are  completed  within  t  hours.  The  third  term  is  the  probability 
that  the  system  was  unavailable  at  the  start  of  the  previous 
mission  and  repair  and  servicing  is  completed  before  the  start 
of  the  current  mission  (a  total  time  of  T+t  hours). 

Solving  for  A(t)  yields 


A(t) 


M(Tj-t) 

1  -  R(T)s(t)  -  mWtf  +  M(T+tf 


The  assumptions  of  constant  mission  and  allowable  mainte¬ 
nance  times,  can  be  relaxed,  and  such  factors  as  malfunction- 
detection  probability  and  repair  efficiency  can  be  included  at* 
the  expense  of  additional  model  complexity.  For  illustrative 
purposes,  however,  the  above  model  will  be  retained. 


By  using  the  preceding  equations  that  relate  availability  to 
maintainability  characteristics,  it  is  possible  to  determine 
maintainability  requirements  from  an  availability  requirement. 
S3.nce  reliability  factors  are  also  involved,  such  determination 
is  best  made  through  a  trade-off  process  wherein  feasibility  and 
costs  are  also  considered  in  selecting  the  appropriate  set  of 
reliability  and  maintainability  requirements.  The  three  cases 
discussed  above  are  considered  with  respect  to  such  trade-offs 
in  the  following  subsections. 

5 .3. 2.1  Point  Availability  Including  Preventive  Maintenance 

The  availability  equation  for  point  availability  including 
preventive  maintenance  is  as  follows: 

. _ MTBM 

R  "  MtEM  +  MDT 

To  obtain  MTBM  (mean  time  between  maintenance),  both  preven¬ 
tive  and  corrective  maintenance  must  be  considered.  It  will  be 
assumed  that  the  mean  time  between  failures  (MTBF)  is  equal  to  9 
and  that  preventive  maintenance  is  scheduled  every  Tp  hours. 

With  failure  time  denoted  by  tf, 

MTBM  =  P[tf  <  Tp]  E  [tf Jtf  <  Tp]  +  F[tf  S  Tp]Tp 

If  it  is  assumed  that  an  exponential  assumption  is  adequate 
for  describing  the  failure-time  distribution,  then 


1  -  e 


-T/e 


MKM  - 


tf  l/0e  "V0  dtf 


l  -  e  p 


-Tp/e 
}  P  T_ 


-T/e  -T/e 

e[l  -  e  p  (1  +  T/e)]  +  e  p  Tp 


/  —L 

ef 1-9  p 


V5  > 
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If  T  /e  is  small,  say  T  /e  s0  or  r 
and  MTBM=P  Tp,  which  J,  „  V®  s0-°5>  then  1  -  e  *>  .  T  & 

actions  occur  much  more  freqSentl?  thSf  preventlve  maintemnce 
actions.  Conversely,  if  1  %  is^Jgl"  sotTa'?  5al?tenanoe 

-T  /Q  p  ^  Ap/e  s 

-L  0  ^  ^  ^ 

corrective  malntenaM^'cceniq''’*^?’  aEal”<  Is  reasonable  since 
maintenance.  C*  000ur3  ““  ®re  frequently  than  preventive 

j-he  mean  downtime  i'MDT)  nam™af 
follows:  In  T  total.  hoursT  TA  ^ent'f  be  estl”>ated  as 
actions  and  1/e  correct've  mi?  P  "tlve  malhter.anoe  (PM) 

pected  to  take  Place?  ae  prSSb!?^6^^^  action3  be  ex- 
Is  preventive  la  then  ?  obablllfcy  that  a  maintenance  action 


T/T 

P  DPMI  =  WJ,  P 

t/tT  + T/& 


B  +  T 


and,  similarly. 


Then 


P  pM]  -  - L 

~ % 


Then,  for  point  availability. 


TV? 


1-  e*V9 


lfc  +  tJ. 

p  ct 


a  choice  of  valuea  fSr'l^V'01’  demonstratlon  is 

If  the  time  interval  t-tlL?*'  3t  ®  a  ^^^bment  on  A? 

r  leased,  l^r™1;:  I-tnT  TV^ 
“  "S»“  • WSSA-JSS.  “«•  lrz  T~« 
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of  downtimes  due  to  preventive  maintenance;  and  while  0  nay  be 
increased  somewhat  and  decreased,  there  is  a  minimum  Tp  value 

below  which  it  would  be  unwise  to  specify. 

A  general  trade-off  relationship  is  difficult  to  develop 
because  the  interrelationships  that  exist  may  be  varied  and 
complex.  Instead,  a  aimple  numerical  example  is  provided  here. 

Assume  thac  there  is  an  availability  requirement  of  O.96. 
Prom  past  experience,  feasibility  analyses,  and  operational  re¬ 
quirements,  the  following  are  reasonable  ranges  for  the  para¬ 
meters  listed: 


0  =  50  -  150  hours 
Tp  =  25  —  75  hours 


M  =  1-3  hours 
pu 

=1-4  hours 


If  the  worst  extreme  is  considered,  i.e.,  0  --  50,  Tp  =  25, 
-  3'  «=t  "  k’  then 


-1/2  T  1 


-1/2 


0.6l 


1  -  e  '  +  ~  (3+2)  0.61  +  O.O67 


=  0.91  , 


indicating  that  the  goal  cannot  be  met  without  careful  attention 
to  requirements.  On  the  other  hand,  under  the  best-case  condi¬ 
tions  of  0=  150,  Tp  =  75,  Rpt  =  Mct  =  1, 


1  -  e 


1  -  e 
-1/2  . 


-1/2 


0.61 


+  1.5/225  C.6l  +  0.007 


=  0.99, 


indicating  that  the  goal  is  feasible  with  an  appropriate  set  of 
requirements . 

_  Assume  now  that  a  more  detailed  analysis  between  T  ,  0,  and 
Npt  yields  the  following  alternatives:  p 


Alternative 

h. 

Max  0 

Min  M. 

I 

25 

150 

1.0 

II 

50 

100 

1.5 

III 

75 

75 

2.0 

iwrnmnm 


The  values  of  M  ^  that  provide  an  availability  of  0.95  are 
determined  from  the  following  equation: 


T  +0 
P 


wnere 

r  =  T  /0. 

r 


The  results  are  as  follows: 

Alternative  Mct 

I  0.72 

II  1.92 

III  1.36 

Because  of  the  initial  restriction  on  of  1  £  Hct  *  4- 

Alternative  1  cannot  be  chosen.  Therefore,  the  choice  is  between 
Hand  III,  and  this  decision  would  depend  on  tne  costs  associated 
with  the  specific  values  of  T^,  6,  and 

This  particular  example  Involves  the  selection  of  a  pre¬ 
ventive-maintenance  schedule  as  well  as  mean  corrective-mainte¬ 
nance  and  preventive-maintenance  times.  Much  more  sophisticated 
models  for  preventive-maintenance  scheduling  have  been  developed, 
and  in  practice  the  procedure  might  be  to  use  one  of  these  models 

to  select  T  and  9  and  then  choose  values  for  M  .  and  M  .  to  meet 
p  ct  pt 

the  availability  goal. 

5 .3 .2. 2  Point  Availability  Excluding  Preventive  Maintenance 

The  following  availability  expression  for  point  availability 
excluding  preventive  maintenance  can  be  used  to  determine  relia¬ 
bility  and  maintainability  requirements: 


a  MTBF 

A  -  MTBF  +  M <m 

If  an  MTBF  goal  has  already  been  established  or  is  quite 
restricted  by  the  state  of  the  ait,  the  3imple  relationship 

MTTR  =  MTBF  (J-l) 


yields  the  required  MTTR  value.  Por  the  more  common  case,  in 
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which  there  are  ranges  of  possible  values  for*  both  MTTR  and  MTHF, 
the  curves  shown  in  Figure  4  provide  a  basis  for  trade-off .  In 
the  figure  use  the  left-hand  vertical  scale  with  the  bottom  hori¬ 
zontal  scale  or  the  right-hand  vertical  scale  with  the  top  hori¬ 
zontal  scale  . 

5 . 3 . 2 . 3  Interval  Availability 

From  the  interval-availability  expression 

_ _ M(T-t-t) _ 

MW  “  i  _  R(T)S(t)  +  R(T)M(t)  +  M(T+t) 

the  maintainability  parameters  of  interest,  are  M(t),  M(T+t),  and 
S(t).  M(T+t)  should  equal  1  since  this  represents  the  probability 
that  maintenance  is  completed  within  the  usual  allowable  time  (tj 
plus  the  mission  time  T.  Then 


A(t) 


-  R{T)S(t)  +  I(T)M(t) 


Since  a  maximum  of  t  hours  is  available  for  servicing  and 
corrective  maintenance,  servicing  should  be  completed  in  much 
less  time  than  t  hours  to  permit  corrective  maintenance  to  take 
place.  In  thi3  case,  a  time  ts  <  t  can  be  choosen  such  that  re¬ 
quirements  are  to  be  placed  on  S(t  )  and  M(t  ),  where  t  plus  t 

s  c  sc 

is  less  than  or  equal  to  t,.  S(t  )  equals  the  probability  that 
servicing  is  completed  within  t  hours,  and  M  (t  )  equals  the 

probability  that  corrective  maintenance  is  completed  within  time 

t  .  Then  M't)  can  be  replaced  by  S(t  )  x  M  (t  )  (assuming  the 
c  see 

independence  of  the  two  associated  events).  The  use  of  this 

product  is  conservative  since  it  is  assumed  that  only  t  hours 

are  available  for  corrective  maintenance  even  if  servicing  is 

completed  earlier  than  t  hours.  The  availability  model  is  then 

s 


v  2  -  R(T)S(ts)  -  S(T)S(t0)Mc(tc) 

Trade-off  curves  relating  R,  S,  and  M  to  A  are  shown  in 
Figure  5.  Again,  cost  and  operational  factors  will  determine 
which  of  the  appropriate  combinations  of  R,  S(te),  and  M„(t„)  to 

specify  for  a  given  availability  requirement.  In  this  example 

the  ^(t,,)  and  M  ft  )  requirements  are  often  called  M  .  -type  re- 
s  c  c  max 

quirements,  which  are  actually  percentile  values  of  the  cumulative 

distribution  function. 
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FIGURE  5 

TRADE-OFF  RELATIONSHIPS  BETVEEN.  S(t^ }  AND  M0(tQ)  TO 
MEET  A  SPECIFIED  INTERVAL  A v AILA5ILITY  REQUIREMENT,  A 


•roaches 


5. 5. 2. 4  Applicability  of  App 

The  three  above-dee.cribed  approaches  for  obtaining  maintain¬ 
ability  requirements  from  an  overall  availability  requirement  are 
only  indicative  of  the  type  of  approach  that  can  be  used.  Several 
simplifying  assumption/*  were  made  in  establishing  the  relation¬ 
ships,  some  possibly  important  factors  were  not  included,  and 
cost  was  given  only  qualitative  consideration.  Therefore,  the 
equations  and  curves,  presented  for  obtaining  specified  values 
must  be  adjusted  to  account  for  factors  that  have  not  been  con¬ 
sidered  adequately  in  this  general  model. 

5.3.3  Realism  of  Specified  Values 

Approaches  simila?  to  those  presented  in  Subsection  5.3.2 
lead  to  a  specified  maintainability  value.  The  next  criterion  is 
one  of  realism.  -It  is  necessary  first  to  establish  what  is  meant 
by  a  realistic  value.  Expressions  such  as  "within  the  state  of 
the  art"  are  commonly  encountered,  and  while  they  do  not  provide 
a  quantitative  assessment,  they  do  convey  the  general  belief  bhat 
that  value  can  be  achieved  by  current  technological  capability. 

Since  maintainability-demonstration-test  requirements  must 
be  established  very  early  in  the  development  program  (often  before 
contract  award)  the  most  logical  approach  to  assessing  realism, 
and  sometimes  even  establishing  the  requirement  if  allocation  from 
higher  levels  is  not  required,  is  to  evaluate  the  maintainability 
performance  of  existing  systems  slmlliar  to  that  under  develop¬ 
ment.  If  the  basic  maintainability  design  is  known  at  the  time 
the  requirement  is  to  be  established,  an  applicable  prediction 
technique  can  be  exercised. 

Whether  historical  data  or  prediction,  or  both,  is  used  for 
assessing  realism,  careful  Judgment  is  required.  If  an  alloca¬ 
tion  leads  to  an  Mc^  value  of  20  minutes  but  a  30-minute  value  was 

observed  for  the  most  similar  existing  system,  can  it  be  con¬ 
cluded  that  20  minutes  is  unrealistic?  The  following  questions 
must  be  considered: 

(1)  How  simi3.ar  are  the  items? 

(2)  Eow  similar  will  the  maintenance  environment  be? 

(3)  Since  the  observed  30-minute  value  is  necessarily  based 
on  a  sample,  what  is  the  lower  confidence  limit  asso¬ 
ciated  with  such  a  mean-value  estimate? 

(4)  Eow  much  maintainability  improvement  can  reasonably  be 
asked  for? 

(5)  Is  there  any  margin  for  increasing  the  20-minute  speci¬ 
fied  value? 
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Again,  the  answers  to  these  questions  and  the  conclusions 
to  be  drawn,  depend  on  individual  circumstances.  To  cneek  for 
realism,  the  prediction  technique  developed  in  this  study,  those 
of  M3X-STD-472,  and  others  presented  in  the  literature  can  be 
used  as  applicable. 

Observed  maintainability  values  of  existing  equipments 
obtained  from  several  sources  are  presented  in  Tables  TfT  through 
XV  to  provide  historical  data  that  can  be  used  as  a  guide  in 
assessing  the  realism  of  a  specified  value.  The  sources  are 
identified  in  the  tables  according  to  the  following  numbered 
references : 

i 

1.  RADC-TDR-63-85,  Vol.  1,  Maintainability  Technique  Study, 

■  Pinal  Technical  Report  (Phase  V),  5  February  19&3.  Pre- 
par’ed  by  RCA. 

2.  RADC-TN6l-l4l,  Maintainability  Measurement  and  Prediction 
Methods  for  Air  Force  Ground  Electronic  Equipment  (Phase 
III  Progress  Report),  15  June  19<5l.  Prepared  by  RGA. 

3.  ARINC  Research  Corporation  Publication  118-4-228,  Main¬ 
tainability  of  Shipboard  Electronic  Systems,  31  Karsh 
1961. 

4.  RADC-TR- 68-398,  Maintainability  Prediction  by  Function, 
Pinal  Report,  August  1968.  Prepared  by  Federal  Electric 
Corporation.  (The  data  from  this  source  were  accumulat¬ 
ed  at  least  in  part  from  the  AFM66-1  reporting  system. 
Since  this  reporting  system  includes  short-duration  ad¬ 
ministrative  delays  in  reporting  man-hours,  the  data  in 
Tables  XII  and  XIU  are  also  contaminated  to  some  degree 
if  they  have  been  derived  totally  or  in  part  from  this 
source.) 

5.  Calculated  from  data  accumulated  by  Federal  Electric  Cor¬ 
poration  under  Contract  F30602-67-C-0194. 

6.  Prom  data  collected  by  ARINC  Research  Corporation  under 
RADC  Contract  No.  F30602-68-C-0047,  Maintainability 
Prediction  and  Demonstration  Techniques. 

Table  XII  presents  observed  maintainability  values  for 
several  classes  of  ground  equipments.  These  data  can  be  used  to 
estimate  the  maintainability  performance  of  a  ground  equipment  if 
no  details  beyond  the  major  functional  classification  are  known. 

Table  XIII  is  an  accumulation  of  maintainability  data  on  a 
number  of  individual  ground  equipments  arranged  by  common  func¬ 
tional  groupings.  If  the  equipment  under  consideration  can  be 


75 


considered  to  be  similar  to  one  of  the  /listed  equipments,  the 
maintained) ility  values  given  in  the  table  may  be  used  in  the 
absence  of  more  precise  estimating  methods. 

The  values  given  in  the  table  represent  total  active  cor¬ 
rective  maintenance,  which  generally  Includes  preparation,  fault- 
location,  fault-correction,  item-obtalnment,  checkout,  and  clean¬ 
up  time,  but  excludes  downtime  due  to  administrative  and  logistic 
delays.  In  most  cases,  the  results  include  maintenance  actions 
for  which  no  trouble  was  found.  The  occurrence  of  such  events  in 
a  demonstration  test  must  be  considered  in  evaluating  these  data. 

Tables  XIV  and  XV  are  similar  to  Table  XIII  except  that  they 
represent  data  on  airborne  equipments  monitored  in  this  study. 

The  observed  maintainability  values  in  Table  XIV  include  mainte¬ 
nance  events  for  which  no  trouble  was  found.  Table  XV  does  not 
include  the  "no  trouble  found"  events  and  may  therefore  be  more 
applicable  for  evaluating  maintainability -demonstration  index 
values. 


TABLE  XII 


MAINTAIKABILm  DATA  FOR  VARIOUS  GROUND  EQUIPMENT  CLASSES 


Equipment  Class 

^CT 

r* 

mct 

"o.95 

Average  MMR 
per  Action 

Data 

Source 

Transceiver 

0.88 

0.36 

3.2 

1.1 

4 

and 

5 

Receiver 

1.83 

0.90 

5.4 

2.3 

4 

and 

c; 

> 

Transmitter 

1.91 

c  80 

6.3 

2.4 

4 

and 

5 

Display /Indie  ator 

1.51 

0.79 

4.4 

1.9 

4 

and 

5 

Data  Processing 

2.23 

1.00 

8.4 

2.8 

4 

and 

5 

Frequency  Power  Supply 

1.11 

0.51 

4.6 

1.4 

4 

and 

5 

Identification 

Recognition 

1.67 

0.80  ' 

5.0 

2.1 

4 

and 

5 

Multiplex 

1.35 

0.37 

4.7 

1.7 

4 

and 

5 

Exciter 

1.75 

0.90 

5.4 

2.2 

4 

and 

5 

Data  Processor 
_ _ _ _ _ _ 

1.19 

0.33 

3.7 

1.5 

4 

and 

5 

I 
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TABLE  XIII 


MAINTAINABILITY  DATA  FOR  VARIOUS 
GROUND  EQUIE.4SNTS 


j  Equipment 

CT 

(Hours) 

/V 

M—n 

CT 

(Hours) 

* 

“0.95 

(Hours) 

n 

HH 

Source 

Radio  Transceivers 

mm 

AN/MPK-1A  (ARC-27) 

1.0 

ii 

AN/MPN-14  (GRA-5A) 

2.0 

4 

AN/GRC-131 

0.2 

4 

AN/GRC-132 

0.2 

4 

AN/GRC-113 

0.3 

4 

AN/GRT-3  and  GRR-7 

1.05 

0.80 

2.8 

1 

AN/GKA-5 

i .  6  c. 

2.13 

2 

Receivers 

AN/FPS-16 

4.4 

4 

AN/FPS-27 

0.61 

0.43 

1.7 

1.25 

6 

ARSR-1B 

1.9 

4 

AN/FPS-30 

1.2 

4 

AN/GSQ-  74B 

0.5 

4 

AN/FPN-47 

2.8 

4 

AN/MPN-14 

1.0 

4 

AN/FRC-102 

i.9 

4 

AN/FRC-96 

2.6 

4 

MM-5Q3A 

1.2 

4 

74A2 

2.1 

4 

an/grc-66 

3.0 

4 

AN/SRC- 12S 

0.9 

4 

AN/TRN-17 

3.0 

4 

AN/SRR- 13A 

2.34 

1.20 

7.0 

1.86 

3.6 

3 

AK/0RR-35A 

1 

5.31 

0.92 

0.43 

6.4 

:3 

AN/SLR- 2 

2.50 

1.60 

6.3 

K  .41 

4.12  j 

i 

L — - , — , - . - _i 

BUwlltfNNi 

- L_ 

--mnri.ii  w  111  ■  ■  J 

mm  Mm  -mm  mt  \mm 

(continued) 
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TABLE  XIII  (continued) 


Equipment  **CT  (  MCT  M0.95 
(Hours)  (Hours)  (Hours) 


Average 

MMH  per  Source 

Action 


Transmitters 


AN/TPQ-18 

AN/FPS-16 

AN/FPS-27 

ARSR-1B 

AN/PPS-30 

AN/GSQ-74B 

AN/FPN-47 

AN/MPN-14(T273B) 

AN/MPN- 14  ( ARC3  ) 

AN/MPN-l4(T867) 

AN/PRC- 102 

AN/FRC-S6 

74A2 

AN/GRC-66 

AN/GRC-113 

AN/FRT-37 

AN/0RN-9C 

AH/TRN-17 

AN/NIRN- 13 

AN/URN-5 

AN/SRT-15 


15.1 

2.8 


30.6  8.5 


AHA‘PQ-18 
AN/PPS-16 
AS/frPS-3  0 
AF/GSQ-74B 
A  J/i'PN-47 
H/15EW- 14 


mkWk 


Equipment 


Display  Indicators 


AN/MPN14(MP49) 

AN/MPNl4(MPA3l) 

74A2 

AN/GSW-5 

AN/GSA-51 

AN/UPA-35 

AN/PSA- 14 

AN/PSA-26 

AN/SPA-4 

AN/SPA-4A 

AN/3PA-8A 

AN/SPA-8C 

VL-1 

ECGM(PPS-27) 


TABLE  XIII  (continued) 


M  «V 

mct  mct 

(Hours)  (Hours)j  (Hours 


Average 

MMH  per  Source 

Action 


0.90 

C.53 

3.2 

I 

2.50 

l.OU 

12.0 

8.57 

4.90 

0.78 

16.67 

1.96 

0.65 

8.0 

6.17 

4.20 

1.47 

7.86 

1.20 

0.43 

0.83 

1.16 

0.94 

3.1 

1.57 


Data  Process! 


AN/TPQ-18 

AN/PPS-16 

AN/PPN-47 

AN/F3T-2 

AN/GSW-5 

AN/GSW-10 

AN/GSA-51 

Computer  (no  mili¬ 
tary  nomenclature) 

Computer  (no  mili¬ 
tary  nomenclature) 

Re corder/Reproducer 


0.93 


0.33 


1.27 


5.1 

2.5 

1.6 
1.27 
0.6 
0.4 
1.50 


3.79 
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IABI£  XIII  (continued) 


Equipment 


Data  Processing 

Punch  Card  System 
(GSA-51) 

Radar 

AN/FPS-6 

AN/FPS-20 

AN/SPS-8A 

AN/SPS-2C 

AN/SPS-12 

AN/SPN-8 

AN/SPN-12 

Navigation 


AN/UfiN-3 

AN/ORD-4 

an/spn-ta 

uency/Power 


AN/PPS-30 

AN/MPN-14 

74A2 

AN/TRN-17 


MCT  mct 
(Hours)  (Hours) 


1.57 

1.11 

3.50 

2.80 

2.50 

1.90 

2.90 


9.60 

7.20 

2.20 


.605 


1.25 

1.26 

1,03 

1.50 

1.43 

2.00 


3.20 

0.50 

1.80 


Average 

MMH  per  Source 

Action 


1.35 


1.58 

L0.5  28.28  5.51 
8.04  3.82 
7-3  22.58  4.17 
7.2  15.21  2.54 

18.91  4.88 


55.29  21.85 
20.26  14.98 
7.62  2.90 


Identification 


AN/UPX-6 

KY-274 
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TABLE  XIII (concluded) 


Equipment 

|T_ 

(Hours) 

**CT 

(Hours) 

“0.95 

(Hours) 

MMH 

■m 

XLCP 

Average 
MMH  per 
Action 

Source 

Ident If 1 cat Ion/ 
Recognition 

KY-248 

1.3 

4 

AN/GPX-8A 

1.7 

4 

AN/MPN14 ( MPA24 ) 

1.5 

4 

AN/UPX-1A 

2.90 

1.30 

18.0 

8.10 

4.50 

3 

Multiplex 

AN/FCC-22 

1.9 

4 

AN/FCC-32 

l.S 

4 

TCS-600 

0.9 

4 

AN/FQC-5 

2.7 

4 

AN/FGC-29 

1.8 

4 

AN/FGC-61 

1.7 

4 

Exciter 

AN/FRO  102 

2.2 

4 

AN/FRC-96 

3.2 

4 

AN/GRC-66 

1.4 

4 

AN/GRC-113 

_ —J 

0.3 

4 

TABLE  XIV 


MAINTAINABILITY  DATA  FOR  AVIONIC  EQUIPMENT 
(NO-TROUBLE-FOUND  ACTIONS  INCLUDED) 


Equipment 


Naviaation/Radlo 


APN-151 

ADF-73 


Electrcmechani- 


mmiter 


ASN-35 

A3N-24 

Radio  Communi- 


iwmM 


ARC- 109 
ARC-90 

Hi Kh- Power 


ar 


APQ-110 
AJQ-20 
APQ-113 
APS- 109 
APN-59 

Low- Power  Navi- 


ransm 


Tiers  an<Tl?e- 


APN-167 

APN-147 

ARN-21 


^CT 

(Hours) 

/V 

mct 

(Hours) 

M0.95 

Average 
MMH  per 
Action 

Source 

0.99 

O.92 

1.80 

1.46 

6 

0.26 

0 . 19 

0.74 

0.26 

6 

c.58 

0.^2 

1.24 

0.81 

6 

0.70 

0.49 

2.29 

0.96 

6 

1.17 

1.00 

2.94 

2.73 

6 

0.51 

0.35 

1.79 

0.82 

6 

1.03 

0.70  • 

2.88 

1.8l 

6 

1.77 

1.47 

4.02 

3.11 

6 

t _ 1 

TO 

O 

0.92 

4.04 

3.04 

6 

1.75 

1.32 

5-79 

4.60 

6 

0.94 

0.55 

3.29 

1.54 

6 

A 

0.64 

0.52 

1.82 

♦ 

1.31 

6 

0.8l 

0.54 

2.72 

1.45 

6 

0.71 

0.50  . 

2.05 

1.15 

6 

AAcur <  AV 


maintainability 

(NO -TROUBLE. 


LATA  FOR  AVIONIC  EQUIPMENT 

PATTVTTN  A  .,nvri>s**«  _ .  . 


Equipment 

flCT 

( Hours ) 

TZ~7~ 

mct  ' 
(Hours) 

M0,95 

Average 
MMH  per 
Action 

Soiirce 

Navigation/Radi c 

■f- - — 

Receiver 

l 

| 

APN-151 

ADE-73 

1  1.30 

0.34 

0.95 

0.27 

2.01 

O.96 

1.63 

0.34 

1 

6 

6 

Electronic  chan  1 - 

cel  Navigation 
ciomnuter 

ASN-35 

ASN-24 

0.63 

0.66 

O.63 

0.64 

1.28 

1.9.6 

1.23 

1.19 

6 

6 

Radio  Common 1- 

cation 

ARC-109 

1.24 

1.07 

3.10 

2.91 

6 

High- Power 

Radar 

apq-ho 

AJQ-20 

AFQ-113 

APN-59 

1.22 

2.46 

l.4o 

1.02 

0.99 

2.20 

0.90 

0.65 

3.65 

4.87 

7.18 

3.46 

2.35 

4.71 

3.34 

1.66 

6 

6 

6 

6 

Low- Power  Navi- 

gationaT  and  " 

tors  and  Re- 
cel vers 

ARN-2.1  / 

0.82  j 

0.63 

2  24 

—  -  1 

1.35  __ 

6 

5.3.^  Applicability  of  Requirements  to  the  Demonstration 
Environment 

I 

In/ the  discussion  of  the  results  of  the  malntalnability- 
demoustration  3urvey  in  Section  II,  it  was  noted  that  the  most 
frequently  cited  difficulty  was  the  difference  between  test  en¬ 
vironment  and  field  environment.  In  an  RADC  study3,  a  compari¬ 
son  of  demonstration-test  results  with  field  operational  results 
for  seven  systems  revealed  wide  discrepancies.  The  operational 
field  MPTR  was  always  greater.  Although  the  field  data  may  have 
been  contaminated  with  some  undesirable  factors  such  as  admini¬ 
strative-time  dealys,  the  observed  differences  are  still  quite 
illuminating. 

It  is  apparent  that  the  closer  the  test  environment  to  the 
expected  field  environment,  the  more  meaningful  the  demonstration 
test,  and  that  every  effort  should  be  made  to  achieve  such  simi¬ 
larity.  Specific  reasons  for  biases  due  to  test  environment 
are  outlined  in  this  section. 

Unless  a  Category  III  type  test  is  to  be  performed,  demon¬ 
stration  environments  will  differ  in  some  respects  from  the  field 
environment.  Because  such  differences  do  exist,  a  maintainability 
demonstration  requiremtnt  based  on  operational  goals  should  not 
be  applied  unless  its  applicability  to  the  demonstration  condi¬ 
tions  is  first  considered. 

As  a  general  principle,  the  specified  value  based  on  opera¬ 
tional  goals  <?nd  conditions  must  be  suitably  adjusted  to  reflect 
the  maintenance  environment  governing  the  demonstration.  Often, 
this  is  a  difficult  principle  to  adhere  to.  With  an  avionic 
equipment,  for  example,  a  certain  amount  of  time  will  be  spent  in 
the  field  Just  reaching  the  equipment  in  the  aircraft,  and  the 
time  to  locate  the  malfunction  and  complete  repairs  and  checkout 
is  a  function  of  this  accessibility  factor.  If  the  demonstration 
test  is  not  to  take  place  in  the  aircraft  (and  this  is  often  the 
case)  there  is  the  quertion  of  whether  the  specified  value  should 
be  adjusted  and  how  much. 

It  might  be  possible  to  construct  a  mockup  to  simulate  the 
actual  conditions,  thus  eliminating  the  need  far  adjustment. 
Generally,  this  type  of  simulation  will  not  be  possible,  and 
field  and  test  conditions  must  be  carefully  analyzed  and  their 
effects  quantitatively  assessed.  Table  XVI  lists  various  factors 
to  be  considered  in  evaluating  the  applicability  of  a  specified 
maintainability  index.  Table  XVII  lists  some  specific  causes  of 
discrepancies  that  are  classified  as  yielding  either  pessimistic 
or  optimistic  results. 


Coppola  and  J.  Deveau,  "Reliability  and  Maintainability  Case 
Histories",  Armais  of  Reliability  and  Maintainability,  Vol.  6, 
196?,  pp.  58!5=58o. 
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FACTORS  APFECTIHO 


My 

ft*  *W?T:l 


!  i»  v  J):r  V  ♦  •*)  J 


F*  #  —  j  , 


Physical  Equipment 

Stage  of  completion 
Similarity  to  production  items 
Physical  location 
Interfacing  equipment 

Test  Location  and  Facllllty 

Lighting  factors 
Weather  factors 
Space  factors 

Test  Team 

Organization 

Training  and  experience 

Indoctrination 


Support  Items 
Tools 

General  and  Special  test  equipment 
Spares  availability 
Technical  manuals 

Operational  Factors 

Mode  of  equipment  operation 
Procedures  for  instituting  maintenance 
Procedures  for  fault  selection 


TABLE  XVII 

. 

i 

CAUSES  OP  DISCREPANCIES  BETWEEN  TEST  AND  FIELD  RESULTS 


A.  Causes  of  Optimistic  Test  Results 

1.  The  demonstration  maintenance  technicians  are  not  rep¬ 
resentative  of  typical  Air  Force  personnel  because  they 
have  more  education  and  training  cr  greater  knowledge  of 
the  equipment  design , 

2.  The  monitoring  situation  imparts  to  the  technician  an 
urgency  not  normally  encountered  in  the  field. 

3.  Known  probable  tasks  are  rehearsed  beforehand. 

4.  Necessary  support  equipment  is  readily  available. 

5.  Observed  times  are  not  contaminated  with  such  factors 
as  administrative  or  logistic  delay,  as  field  results 
sometimes  are. 

6.  Bifficult-to-iaolate  faults  Buch  as  intermittencies  and 
degradation  failures  are  not  simulated. 


B.  Causes  of  Pessimistic  Test  Results 

1.  The  technicians  are  not  familiar  with  the  equipment 
and  have  not  acquired  the  necessary  experience  for 
rapid  fault  isolation. 

2.  Field  and  procedural  modifications  to  reduce  maintenance 
time  have  not  yet  been  made. 

3.  Initial  manuals  may  be  incomplete  or  require  revision. 

4.  The  monitoring  situation  can  adversely  affect  the  tejh- 
nician’s  performance. 
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5.4  RISK  ASSIGNMENT 


5.4.1  General 


There  are  generally  two  risks  Involved  la  a  deaonBt ra¬ 
tion  test : 

(1)  Producer’ t  i-luk,  a  —  the  probability  of  rejection  if 
tie  maintainability  characteristic  is  at  the  desired 
level 


(2)  Consumer's  risk,  p  —  the  probability  of  accepts***  if 
the  maintainability  characteristic  is  at  the  ■juflwua 
acceptable  (or  undesirable)  level 


Ideally,  a  and  p  would  be  equal  to  seroj  granting  that  thisr 
Is  impossible,  very  small  values  of  o  and  p  —  o»  the  order  of 
0.001  —  are  desirable.  Such  naall  values  are  also  imprastissl 


since,  a a  discussed  in  Section  XXI,  the  selection  of  o  end  p 
associated  with  the  HQ  and  values  for  maintainability  3£ota£es 


the  sample  slse.  For  a  and  p  on  the  order  of  0.001,  ffMMfrt^ 
far  exceeding  available  test  resource*  will  usually  he  fe&irsd* 


xckL  ovau-awic  vcov  iTCttU'VMTCVll  nuu  WWKUJ  URniiHa 
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For  example,  consider  a  test  cf  the  mean  of  a  logeormal  dia» 
tributlon  such  as  the  following: 


30  minutes 


H^:  u  -  Ux  -  45  minutes 


As  shown  in  Section  VU  (Test  Nraaber  1),  the  necessary  sample 
size  for  tnis  teat  is  given  by  the  iviuation 


i5£LL?^L  (e^.x, 


-  V-J* 


where  Z  end  Z_  are  the  nonrial  deviates  corresponding  to  %fts 

*  _  %  JLU  •»  .  •  flK  . 


(1  -  o)?h  and  (1  -  p)^  percentile  of  a  normal  (0,  1)  distribution 


and  o2  3-j  tne  variance  of  the  logarithm  of  maintenance  ti»s.  If 
a  =  p  and  c2  «  1  are  assumed,  then  ~vv. 


zf  (30  4-  45  f  ,  i  -• 
--fts' -  30)3- 


Figure  6  shows  the  relationship  between  n  and  the  risk 
values.  In  the  figure*  it  can  be  seen  that  if  a  =  0  =  o.lo,  JG 
observations  (a  reasonable  number)  are  required.  If  a  and  0  are 
reduced  to  0.01,  about  230  observations  are  necessary,  and  for 
h  •  0  «*  0.001,  a  sample  else  of  more  than  400  Is  called  for. 

Host,  development  budgets  and  schedulas  will  not  allow  for  a 
test  requiring  400  sample  observations  even  if  the  observations 
are  to  be  simulated.  In  fact,  even  a  sample  size  of  JO  my  tax 
available  resources,  and  for  this  illustrative  case,  risks  on  the 
order  of  0.15  or  0.20  may  be  necessary. 

It  is  not  necessary,  of  course,  for  a  to  equal  0.  If,  for 
example,  the  need  for  the  equipment  is  great  and  a  43-minute  mean 
tint  to  repair  can  be  tolerated  (perhaps  with  later  improvement 
by  modification  and  appropriate  training,  manning,  ana  support 
planning),  the  0  risk  may  be  set  at  a  higher  level,  say  0.25. 


The  choice  of  a  and  0  is  also  one  involving  trade-offs, 
from  a  decision-theory  viewpoint,  the  trade-off  can  be  normalized 
to  a  cos4:  criterion  based  on  the  following: 

|l)  Cost  of  testing  (sample  size) 

2,  Coa'c  of  rejecting  good  equipment 
3)  Cost  of  accepting  poor  equipment 

While  (l)  can  generally  be  costed  in  terms  of  manpower, 
facilities,  and  time,  (2)  and  (3)  are  more  difficult  to  assess 
quantitatively .  Assuming  that  prior  information  is  available 
for  estimating  at  least  relative  values  associated  with  the  three 
costs,  two  simplified  approaches  employing  decision-theory  con¬ 
cepts  for  selecting  u  and  0  are  discussed  below,  for  convenience 
the  maintainability  characteristic  of  interest  will  be  denoted  by 
K,  and  specified  S L  and  H,  values  by  MU  and  ML,  respectively. 


5-4. 2.1  jSnlaax  Criterion 

The  mini max  criterion  is  used  when  it  is  desirable  to 
eattnasely  high  costs.  In  order1  to  use  this  criterion 
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for  a  given  calcination  of  a  and  p,  say  (a 
following:4 

(1)  ^(M0)-?0«i  + 

(2) 

(3)  Z*.  -  Bax  fit,  .(HJ, 


,  compute  the 


Test  costa  associated  with  (a 
if  M  -  (k  =  0  or  1) 

Total  cost  if  M  *  M.  (k  =  0,  : 


Generally  will  be  a  function  of  the  sample-size  re¬ 

quirements  dictated  by  the  a.,  p  pair  and  will  not  depend  on  M 
except  for  sequential  test s,  f or*5 which  the  average  value  of  n 
given  M  *  1^  can  be  used. 

The  a,  p  risk  pair  to  select  is  that  which  has  the  minimum 
value  of  T^j.  Ifr  this  criterion  the  selected  risks  are  such  that 

the  maximum  possible  costs  are  minimized. 


gxample:  Consider  the  illustrative  test  discussed  above, 
lor  simplicity,  assume  that  the  values  of  a  and  p  to  be  considered 
are  restricted  to  0.05,  0.10,  0.20.  Some  possible  risk  pairs  and 
associated  sample  sizes,  from  the  previous  equation,  are  as 
follows: 

Pair  (i,j)  a  p  n. , 


Jk/Pheue  equations  are  based  on  the  assumption  that  no  costs  are 
associated  with  an  accept  decision  if  M=Hq,  or  reject  decision 
it'  ll  *  JS.,  except  for  the  test  costs.  u 
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In  this  procedure,  the  risks  are  selected  to  minimize  the 
expected  costs. 

Example?  Assume  that  it  can  be  reasonably  estimated  from 
past  performance  data,  in  conjunction  with  evaluation  of  the 
maintainability  program  efforts,  that  PQ  =  0.70,  P.  ■=  0.30.  The 
values  associated  with  this  prior  distribution  are1as  follows. 


index 

Risks 

Expected 

eosts, 

Eij 

i 

J 

a 

0 

1 

1 

O.03 

0.05 

$17,Po6 

1 

2 

0.05 

0.10 

12 ,  3I9 

1 

3 

0.05 

0.20 

9,5l4* 

2 

1 

0.10 

0.05 

15,316 

2 

2 

0.10 

0.10 

11,600 

2 

3 

0.10 

0.20 

9,836 

3 

1 

0.20 

0.05 

15,22S 

3 

2 

0.20 

0.10 

12,904 

3 

3 

0.20 

0.20 

12,300 

•Minimum  Value. 


Prom  the  above  listing,  it  is  seen  that  the  risk  u  «  O.05, 

P  -  0.20  minimizes  expected  cost.  If  the  prior  probabilities 
were  pq  *  P1  “  °*5°>  the  pair  o  *  0.10  p  «  0.  20  would  be  optimal. 

With  the  prior  estimates  of  PQ  and  P.,  the  expected  cost  without 
testing  can  also  be  evaluated;  If  no  testing  is  performed  and 
the  equipment  is  to  be  accepted  upon  delivery,  the  expected  cost 

is  simply 


(P1)(C1)  =  (0.30)(40,000)  =  $12,000. 

For  this  example,  the  decision  not  to  test  is  unwise.  How¬ 
ever,  where  testing  is  quite  costly  and  past  performance  indicates 
a  high  probability  of  a  satisfactory  product,  this  type  of  evalu¬ 
ation  might  indicate  that,  from  the  viewpoint  of  economy,  little 
or  no  testing  is  the  preferred  choice. 

5. 4.2. 3  Summary  of  Decision-Theory  Approach 

The  two  decision-theory  approaches  described  above  might  be 
criticized  on  the  basis  that  only  the  Hq  and  H.  values  for  M  are 
considered.  More  extensive  procedures  can  be  Used,  but  they  re¬ 
quire  prior  information  and  cost  relationships  tha;;  are  not  gen¬ 
erally  available. 

In  defense  of  the  procedure,  it  can  be  said  that  ic,.,  con¬ 
ventional  sampling  procedures,  in  which  a  and  p  are  more  or  less 
arbitrarily  chosen,  two  levels  of  maintainability  are  also  con¬ 
sidered.  Moreover,  the  Mq  and  values  and  their  associated 
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SECTION  VI 


SELECTION  OF  MAINTENANCE-TASK  3AMPIE 

6.1  GENERAL 

There  are  two  basic  approaches  for  selecting  a  sample  of 
maintenance  tasks  for  the  demonstration: 

(1)  observe  maintenance  tasks  as  they  occur  naturally  in 
an  operational  or  simulated  operational  situation 

(2)  Induce  faults  in  the  system  and  observe  ‘-•he  mainten¬ 
ance  actions  to  corr  jet  these  faults 

The  terms  "natural  failures"  and  "fault  inducement"  will 
be  used  to  distinguish  the.se  two  approaches. 

For  the  fault -inducement  approach,  a  decision  must  be  made 
on  the  type  of  sampling  procedure  to  be  used.  The  usual  choice 
is  between  stratified  sampling  and  simple  ran^im  sampling. 

In  this  section  guidelines  are  offered  for  evaluating  the 
applicability  of  the  two  basic  approaches,  obtaining  maintenance 
task  samples,  and  choosing  the  appropriate  sampling  design  and 
procedure . 

6.2  NATURAL  VERSUS  INDUCED  FAILURES 

In  most  cases,  the  cho3.ce  of  natural  or  induced  failureis 
must  be  made  early  in  the  development  program  since  the  natural 
failure  approach  can  be  used  only  if  the  program  schedule  allows 
enough  time  to  obtain  the  required  number  of  maintenance  tasks. 
This  allowable  time  is,  of  course,  related  to  reliability. 

If  0  is  the  MTBF  of  an  equipment,  the  average  number  of 
operating  hours  that  will  be  required  to  yield  n  failure  occur¬ 
rences  is  ne  .  For  equipments  with  MTBFs  of  hundreds  of  hours 
and  required  sample  sizes  of  30  to  70,  the  number  of  required 
equipment  operating  hours  can  easily  exceed  10,000  (e.g.,  50 
samples  from  an  equipment  with  an  MTBF  of  200  hours).  If  three 
such  equipments  are  available  for  test  and  the  equipments  are 
operated  16  hours  a  da>,  an  average  of  more  than  200  days  would 
be  required  to  complete  the  demonstration. 

Because  of  time  requirements  of  this  magnitude,  most  main¬ 
tainability  demonstrations  are  based  on  the  fault -inducement 
approach,  by  which  the  demonstration  can  be  completed  in  a  few 
days . 


With  respect  to  realism  and  applicability,  the  natural- 
failure  app-^oach  is  clearly  the  preferred  choice.  The  major 
disadvantage  of  inducing  faults  is  that  there  is  no  guarantee 
that  these  faults  are  representative  of  those  which  will  be 
seen  in  operation.  In  addition,  the  f suit- inducement  plan  pro¬ 
vides  information  for  rehearsal,  which  will  naturally  bias  the 
test  in  a  direction  that  is  favorable  to  the  contractor.  Un¬ 
fortunately,  there  have  been  Indications  that  such  rehearsals 
have  taken  place  in  demonst-'ation  tests. 

Because  of  the  problem*  associated  with  fault  inducement, 
the  following  general  recommendations  are  made: 

•  If  the  schedule  can  allow  for  natural  failures,  then  this 
type  of  sampling  procedure  is  preferred.  Category  III 
type  tests  will  logically  fall  within  this  class. 

•  If  the  complete  demonstration  cannot  be  completed  with 
only  naturally  occurring  failures,  a  combination  of  the 
two  approaches  should  be  used.  One  possibility  is  to 
take  advantage  of  the  reliability-demonstration  test  and 
include  in  the  sample  the  maintenance  times  needed  to 
correct  faults  that  occurred  In  the  reliability  demon¬ 
stration.  Close  coordination  between  the  two  test 
groups  will  be  required. 

•  If  a  natural-failure  test  cannot  be  conducted,  any  natural 
failures  that  do  occur  during  the  induced-failure  test 
should  be  included  in  the  sample. 

• 

6.3  FAULT-IKDUGEKSOT  PROCEDURES  , 

The  first  criterion  for  judging  the  suitability  of  a  fault- 
inducement  procedure  is  whether  it  leads  to  a  series  of  main-1 
tenonee  t&eka  that  are  representative  at  the  level  of  mainten¬ 
ance  specified.  Thus  the  maintenance  tasks  generated  by  fault 
inducement  ahould  be  representative  with  respect  to  the  follow¬ 
ing: 

•  Engineering  and  maintenance  factors  such  a a  symptom  in¬ 
dications  and  required  repair  procedures 

•  Frequency  of  occurred', 

The  latter  will  be  considered  in  Subsection  6.4,  which 
describes  the  sampling  procedure. 

Frequently  used  methods  for  inducing  a  fault  are  to  tape  a  ,  - 
connector  pi?:  or  disconnect  a  lead  to  simulate  an  open,  ground¬ 
ing  a  wire  or  pin  to  simulate  e  short,  inserting  a  known  faulty 
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part  or  blown  fuss,  end  removing  a  circuit  card  or  wire. 

The  method  used  should  not,  of  course,  provide  the  techni¬ 
cian  with  Information  that  he  would  net  normally  receive  under 
actual  maintenance  conditions.  Taping  a  pin,  therefore,  vxrald 
not  be  an  acceptable  method  if  the  level  of  maintenance  was 
such  that  the  technician  could  easily  spot  this  type  of  alssa- 
lated  fault.  If,  however,  maintenance  were  at  the  module  level 
end  the  pine  were  internal,  such  a  method  would  be  acceptable. 

More  than  avoiding  obvious  circulations  le  required  for 
adequate  fault  inducement.  A  review  of  p*-»t  maintainability 
demonstrations*  indicated  that  the  fault-inducement  methods  in¬ 
volve  a  great  number  of  disconnects,  card  removals,  wire  ground¬ 
ings,  and  the  like  to  simulate  either  shorts  or  opens  with  lit¬ 
tle  or  no  regard  to  other  type  failures. 

These  methods  are  relatively  easy  to  accomplish;  they  can 
he  controlled  ec  that  the  equipment  is  not  damaged  and  the  In¬ 
duced  fault  can  be  easily  corrected  upon  completion  of  the  main¬ 
tenance  observation.  However,  they  may  not  lead  to  a  represen¬ 
tative  set  of  maintenance  tasks;  Failures  resulting  from 
out, -of -tolerance  or  degradation  conditions  or  intenaittencies 
and  those  of  a  secondary  nature  leading  to  a  multiple  failure 
occurrence  usually  cannot  be  simulated  by  these  simple  methods. 

There  are  several  possible  approaches  for  inducing  non¬ 
cat  a  atrophic  Tenures: 

•  Replacement  of  a  good  part,  circuit,  or  assembly  with 
an  identical  item  with  an  appropriate  type  failure 

•  Insertion  of  extra  nondetfectable  parts  such  as  a  by¬ 
pass  resistor  to  simulate  an  out-of-tolerance  condition 

•  Deliberate  misalignment 

•  Use  of  cold- solder  joints  to  induce  internal ttencies 

Consideration  should  be  giver,  to  including  failures  that 
canno*  be  attributed  to  piece-part  inherent  reliability  juch  as 
nickfcd  insulation,  broken  wires,  and  items  abused  by  operation 
or  through  maintenance  (e.g.,  bent  pins).  Secondary  failures, 
which  are  a  result  of  a  primary  failure,  must  be  considered  if 
their  frequency  of  occurrence  is  not  negligible . 

The  strongest  argument  for  avoiding  noncata atrophic  fail¬ 
ures  is  that  they  are  difficult  to  Induce  by  simulation,  and 
this  cannot  be  denied .  However,  the  cost  of  such  failure  in¬ 
ducement  should  be  small  relative  to  the  total  cost  of  the  dem¬ 
onstration.  More  important,  the  additional  investment  for  in¬ 
cluding  more  than  just  open-  and  shert-type  failures  provides 
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a  more  representative  set  of  tasks  and,  therefore,  eliminates 
a  possibly  serious  biasing  factor 

One  approach  to  achieving  this  type  of  noncatastrophic 
representation  is  for  the  oontrrctor  to  retain  parts,  circuit 
cards,  assemblies,  etc,,  that  have  been  rejected  during  develop¬ 
ment,  reliability,  and  quality-control  tests  for  possible  use  in 
the  maintainability  demonstration.  This  is  particularly  impor¬ 
tant  for  failures  that  are  difficult  to  simulate,  such  as  lnter- 
aittenoles  and  instability. 

From  the  foregoing  discussion,  it  is  apparent  that  the 
types  of  faults  to  be  simulated  and  the  methods  of  simulation 
roust  be  considered  early  iii  the  development  program.  Such 
early  planning  will  make  it  possible  to  use  the  information 
from  development-type  teste  to  achieve  realism  through  effective 
simulation.  Because  of  the  intricac5**n  that  may  be  involved  in 
simulating  certain  types  of  malfunctions,  the  guidance  of  de¬ 
sign-engineering  and  reliability  personnel  will  generally  be 
required  in  planning  the  fault -inducement  procedures.  The  re¬ 
sults  of  failure-mode  and  effects  analyses  (FMBA),  reliability 
predictions,  and  reliability  tests,  are  particularly  applicable. 

6.4  IESIGN  OF  MAINTENANCE-TASK  SAMPLE 

Randomness,  lack  of  bias,  representativeness,  and  effi¬ 
ciency  are  several  of  the  criteria  for  evaluating  a  sample  de¬ 
sign.  In  an  experiment  such  as  testing  the  efficacy  of  a  drug 
on  laboratory  animals  or  polling  a  relatively  known  and  stable 
population  on  a  political  question,  the  use  of  ouch  criteria  is 
possible,  meaningful,  and  prudent.  In  maintainability  demon¬ 
stration,  however,  the  population  of  interest,  maintenance- task 
tl^es,  does  not  usually  exist  at  the  time  of  the  test,  and  the 
anomaly  of  sampling  from  a  nonexistent  population  certainly 
complicates  sample  design. 

Sampling  from  a  population  whose  specific  characterlsticfi 
have  not  been  previously  observed  3s  not  unique,  however.  For 
example,  it  Is  possible  to  estimate  the  number  of  different 
types  of  fish  in  a  newly  discovered  lake  from  a  sample  designed 
by  stratifying  the  lake  by  areas,  if  general  information  is 
available  on  the  habitats  of  different  fish.  Care  must  be  taken 
to  eliminate  biases  in  methods  used  for  catching  the  fish,  time 
of  day,  season,  etc.  —  that  is,  the  procedures  used  should  not 
result  in  obtaining  one  species  of  fish  in  a  greater  proportion 
than  actually  exists  in  the  population. 

Similarly,  the  basic  objective  in  obtaining  samples  from 
naturally  occurring  failures  or  from  fault- inducement  procedures 
should  be  to  yield  unbiased  estimates  of  the  maintenance  para¬ 
meter  of  interest.  It  is  probably  impossible  to  avoid  bias 
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completely.  Tor  example,  the  Inducement  of  a  fault  that  can 
seriously  damage  the  equipment  is  almost  always  prohibited. 
However,  possibly  serious  biasing  factors,  such  as  inducing 
only  easily  repaired  faults,  should  be  closely  controlled.  If 
the  direction  Hud  magnitude  of  unavoidable  remaining  biasing 
factors  Is  known,  appropriate  adjustments  to  the  specified 
values  may  have  to  be  made,  as  discussed  previously. 

6.4.1  Sample  Design  for  s  Batursl-Tftilures  Test 

Tor  demonstration  based  on  natural  failures,  there  is  lit-* 
tie  flexibility  in  sample  selection.  As  failures  occur  or  pre¬ 
ventive  maintenance  actions  are  called  for,,  observers  record 
the  appropriate  maintainability  characteristic  when  the  main¬ 
tenance  action  is  initiated.  The  small  number  of  available 
equipments  and  test  environments  and  limited  test  time  will 
generally  preclude  the  use  of  sophisticated  sampling  precatfrxr*# 
and  require  following  an  "observe  everything  we  can”  philo  .%  . 

Those  responsible  for  sample  design  should  review  th»  -*v- 
ditions  under  which  the  natural  failures  will  occur.  If  th 
analysis  Indicates  that  certain  types  of  actions  will  be  un¬ 
likely,  as  might  be  the  case  If  the  operational  mode  under  test 
does  not  require  using  a  portion  of  the  system,  then  som  fault 
Inducement  may  be  necessary.  Therefore,  "filling  out  the  sample" 
should  be  considered  for  a  natural-failures  test. 

To  aid  further  in  determining  representativeness.  It  io 
recommended  that  a  list  of  the  various  possible  maintenance 
tesks  be  developed,  possibly  grouped  by  similarity  of  required 
procedures  and  expected  task  times.  The  frequency  of  occur¬ 
rence  of  each  group  must  then  be  estimated  on  the  basis  of  such 
factors  as  number  of  items  in  the  system  that  can  lead  to  per¬ 
formance  of  the  particular  task  or  group  of  tasks,  the  relia¬ 
bility  of  these  items,  and  the  operating  duty  cycle. 

Such  a  list  will  then  provide  an  estimated  relative  fre¬ 
quency  of  occurrence  of  various  tasks,  which  can  be  used  to 
evaluate  possible  biases  in  the  observed  sample.  Since  the 
list  must  be  developed  partly  on  the  basis  of  estimates,  such 
as  failure  rate,  there  is  no  guarantee  that  the  checklist  is 
"right"  and  the  sample  "wrong".  However,  such  a  lint  can  pro¬ 
vide  a  warning  signal  for  large  discrepancies,  which  should 
then  be  investigated  in  greater  detail. 

Since  the  development  of  such  a  list  is  the  basic  approach 
to  designing  a  faul- inducement  sample,  the  details  are  presented 
in  the  following  subsection,  which  deals  with  that  method  of 
sample  selection . 
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6.4.2  Sample  Design  for  a  Fault -Inducement  Test 

6. 4. 2.1  Simple  Random  Versus  Stratified  Sampling 

The  basic  choice  In  designing  a  sampling  procedure  for  in- 
duclng  faults  is  between  simple  random  sampling  and  stratifiec. 
sampling. 

A  simple  random  sample  is  one  in  which  all  possible  samples 
of  n  units  out  of  the  population  have  an  equal  chance  of  being 
chosen.  A  stratified  random  sample  is  one  In  which  the  total 
population  is  divided  into  subpopulations  or  strata  and  sample 
sizes  for  each  stratum  are  then  determined  according  to  selected 
criteria.  Random  sampling  is  then  performed  within  each  sub- 
population. 

Because  there  is  uo  physical  population  as  such  from  which 
to  sample,  it  is  necessary  first  to  develop  a  hypothetical  popu¬ 
lation  of  maintenance  tasks.  This  hypothetical  population  pro¬ 
vides  the  basis  for  sampling  by  the  fault -inducement  procedure 
for  both  random  and  stratified  sampling. 

A  simplified  scheme  for  presenting  this  hypothetical  popu¬ 
lation  is  shown  in  Table  XIX.  The  maintenance-cask  groups  rep¬ 
resent  all  the  different  types  of  maintenance  tasK2  that  may  be 
performed,  ranging  from  simple  adjustments  to  complicated  mech¬ 
anical  repairs.  Similar  tasks  are  usually  grouped  together. 

The  expected  number  of  task  occurrences  within  a  maintenance- 
task  group  can  be  estimated  by  the  equation 


where 

*w 


T  Is  the  average  mission  time 

The  sum  of  the  E^'s  is  the  expected  number  of  maintenance 

act lone  in  T  hours,  and  this  is  used  to  obtain  the  relative  task- 
occurrence  probabilities,  p^.  Details  of  developing  such  a 

table  are  presented  in  Subsections  6. 4. 2. 2  and  6. 4. 2. 3.  Table 
XIX  will  be  used  here  to  discuss  the  distinction  between  simple 
and  stratified  sampling. 


Ei  “  5  T 


is  the  failure, rate  of  the  jth  item  in  the  Ith 


group 


,th 


is  the  duty-cycle  factor  for  the  J  item  in  the 
ith  group  (0  <  *  1.0) 
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table  XIX 


MAINTEKANC2-TASE  POFOXATICN 


Maintenance-task 

Group 

Expected  Number 
of  Occurrences 
in  T  Hours 

Relative  Maintenance- 
Task  Population 

1 

Ei 

Ej^/E  m 

2 

E2 

• 

e  pg 

• 

i 

• 

Ei 

• 

m 

E^/E  » 

• 

s 

• 

Eb 

• 

Vs  * 

s 

s 

E 

To  select  a  simple  random  sample  of  size  r.,  n  random  num¬ 
bers  between  0  and  1  can  first  be  drawn,  such  as  from  a  random- 
number  table.  If  a  selected  random  number,  x,  Is  In  the  inter¬ 
val  0  <  x  s  p-^,  a  fault  generating  a  task  In  the  first  group  Is 

Induced.  If  it  is  In  the  Interval  p^  <  x  s  (p-j^  +  p2),  the 

second  type  of  task  is  generated.  [(p1  +  p2)  <  x  *  (pj  +  p2  +  p^)  ] 

defines  the  interval  for  generating  the  third  type  of  task,  and 
sc  forth.  By  this  procedure,  all  possible  samples  of  size  n  have 
an  eq’^l  chance  of  being  observed. 

The  most  commonly  used  method  of  stratified  sampling  in 
maintainability  demonstration  is  proportional  stratified  sam¬ 
pling.  In  this  method,  the  sample  size  from  each  stratum  (e.g., 
maintenance- task  group)  is  proport iona.l  to  the  population  size 
of  the  stratum.  Thus  if  there  were  five  strata  with  relative 
population  sizes  or  5,  20,  20,  25,  30,  and  a  total  sample  of  50 
observations  were  to  be  made,  2  or  3  observations  would  be  se¬ 
lected  from  the  first,  or  smallest,  stratum,  5  from  the  second 
and  third,  .10  from  the  fourth,  12  or  13  from  the  next,  and  15 
from  the  largest  stratum. 
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It  Is  noted  that  for  a  simple  random  sample,  it  is  unlike¬ 
ly  that  such  a  “natural”  sample-size  distribution  would  occur, 
but  as  the  total  sample  size  increases,  the  simple  randan  and 
proportional  stratified  samples  will  tend  to  coincide.  For  a 
small  sample,  however,  it  is  quite  possible  that  a  sample  from 
a  specific  stratum  will  not  be  observed  in  a  random  sample. 

For  example,  if  the  above  relative  population  sizes  are  used, 

with  a  tonal  sample  size  of  20,  there  is  a  (0.95) 20  *  0.112 
probability  that  no  samples  from  the  smallest  stratum  will  be 
observed,  while  a  proportional  sample  would  dictate  that  one 
such  observation  be  drawn. 

The  most  common  applications  of  the  theory  of  hypothesis 
testing  are  based  on  the  assumption  cf  simple  random  sampling. 
On  the  other  hand,  stratified  sampling  is  often  employed  in 
large-scale  survey  work  to  take  advantage  of  the  greater  pre¬ 
cision  that  such  a  procedure  offers  in  obtaining  estimates  of 
population  parameters. 

Historically,  proportional  stratified  sampling  has  been 
used  in  maintainability  demonstration  (MIL-STl£471,  Appendix  A, 
Task  Selection  Method),  probably  because  identification  of  the 
maintenance- task  population  requires  a  form  of  stratification 
and  because  stratified  sampling  can  ensure  that  at  least  one 
sample  observation  from  each  selected  stratum  will  be  included 
in  the  sample,  thus  providing  some  psychological  assurance  of 
reprekentatxvenesa . 

There  are  several  advantages  to  stratification.  In  some 
cases,  a  hypothesis  concerning  one  portion  of  the  system  or  one 
type  of  maintenance  may  be  of  interest  and,  therefore,  there 
may  be  a  sample-size  requirement  for  each  of  the  appropriate 
subpopulations.  For  example,  if  a  maintainability  test  is  con¬ 
cerned  with  total  maintenance  man-hours,  it  may  be  advisable  to 
treat  corrective-  and  preventive-maintenance  actions  as  two 
subpopulations  so  that  inferences  can  be  made  for  each  as  well 
as  for  the  total  population  of  maintenance  actions. 

A  second  advantage  is  that  there  may  be  cases  in  which 
administrative  considerations  will  dictate  the  use  of  stratifi¬ 
cation.  For  example,  in  conducting  a  maintainability-demonstra 
tion  test,  it  may  be  advisable  to  consider  the  electronic  and 
mechanical  portions  separately,  and  the  sampling  approach  in 
each  may  differ,  thus  nece  stating  a  stratified  approach. 

The  third  major  advantage  of  stratification  is  statistical 
By  stratifying  a  heterogeneous  population  into  homogeneous 
strata,  the  variation  within  each  st return  is  minimized  so  that 
an  estimate  of  each  stratum  mean  can  be  obtained  with  a  rela¬ 
tively  small  sample .  These  strata  means  are  then  combined  so 


that  the  overall  mean  estimate  has  smaller  variance  than  that 
cf  simple  random  samples.  This  concept  can  be  extended  to  ob¬ 
tain  a  stratified  plan  that  minimizes  testing  costs  where  costs 
of  testing  vary  over  the  strata  elements. 

The  major  disadvantage  of  stratification  Is  that  while  It 
Is  an  effective  method  for  increasing  the  precision  of  sample 
estimates,  it  introduces  considerable  complexity  in  the  neces¬ 
sary  analytical  procedures  for  hypothesis  testing.  For  ex¬ 
ample,  if  the  hypothesis  under  test  concerns  the  mean  of  a 
normal  distribution,  the  well  known  "tn  test  is  usually  applied 
under  simple  random  sampling.  When  stratified  campling  is  used, 
such  simple  application  may  be  impossible  since  even  though  the 
overall  population  is  normally  distributed,  the  distribution 
within  each  stratum  may  be. far  from  normal.  Most  textbook  dis¬ 
cussions  of  confidence-interval  and  hypothesis-test  inference 
are  therefore  based  on  the  assumption  of  simple  random  sampling. 

The  procedures  cxirrently  used  to  analyze  the  results  of 
maintainability-demonstration  tests  employing  proportional  stra¬ 
tified  sampling  are  based  on  the  assumption  that  only  simple 
random  sampling  has  been  performed. 

As  mentioned  above,  for  large  sample  sizes  a  proportional 
stratified  sample  and  a  simple  random  sample  will  yield  essen¬ 
tially  similar  results.  Therefore,  for  large  sample  sizes  (at 
least  50  observations)  and  a  small  number  of  strata  (e.g.,  no 
more  than  10),  the  use  of  simple  random- sampling  analysis  pro¬ 
cedures  for  a  proportional  stratified  sampling  can  be  considered 
acceptable  if  one  is  willing  to  forego  the  greater  precision 
generally  offered  by  stratified  sampling. 

It  should  be  noted,  however,  that  for  a  stratified  random 
sample,  while  all  elements  in  the  population  may  have  equal 
\  chances  of  appearing  in  the  sample,  the  sample  observations  are 
'  not  independent  with  regard  to  order.  For  if  (n-1)  observa¬ 
tions  have  been  made,  the  last  item  must  be  selected  from  a  par¬ 
ticular  stratum,  whose  identity  depends  on  the  previously  ob¬ 
tained  sample  observations.  This  type  of  dependence  violates 
one  of  the  principal  assumptions  of  standard  sequential- sampling 
plans;  therefore,  a  stratified  random  sample  is  generally  not 
appropriate  for  sequential  testing  as  commonly  used. 

Tc  summarize,  stratified  sampling  will  yield  more  efficient 
tests  than  simple  random  sampling  provided  the  following  con¬ 
ditions  are  satisfied: 

•  There  is  a  good  basis  for  stratification. 

•  The  variance  within  each  stratum  Is  small. 
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•  The  strata  population  sizes  are  known. 


Appropriate  analytical  procedures  are  available. 


If  simple  random- sampling  analytical  procedures  arc  em¬ 
ployed,  the  main  advantage  of  sampling  by  proportional  strati¬ 
fication  is  psychological  in  that  assurance  is  provided  for 
Including  samples  from  each  of  the  stratified  populations.  It 
is  believed  that  this  is  the  major  Justification  for  stratified 
sampling  in  maintainability  demonstration. 


Table  XX  provides  guidelines  for  using  simple  randoh  and 
stratified  sampling  by  presenting  a  summary  comparison  for 
several  major  factors. 


6. 4. 2. 2  Stratification  by  Maintenance-Taek  Groups 


As  indicated  above,  regardless  of  whether  simple  random  or 
stratified  sampling  is  used,  a  hypothetical  maintenance-task 
population  must  first  be  developed  when  faults  are  being  induced. 
Identifying  the  maintenance-task  groups  is  the  first  step  to¬ 
wards  this  goal,,  For  discussion  purposes,  the  terms  mainten¬ 
ance-task  group  and  maintenance-task  stratum  are  used  syr.ono- 
mously . 


For  practical  reasons,  stratification  or  grouping  is 
usually  performed  for  both  simple  random  sampling  and  stratified 
random  sampling.  The  difference  between  the  two  is  that  for 
stratified  sampling,  the  number  of  samples  to  be  taken  from 
each  stratum  is  predetermined,  while  i'or  simple  random  sampling, 
the  number  of  samples  to  be  taken  from  each  stratum  is  a  random 
variable . 


For  discussion  purposes,  comments  will  be  restricted  to 
stratified  sampling,  since  they  will  also  generally  apply  to 
simple  random  sampling  when  task  selection  by  fault  Inducement 
is  being  considered. 


The  first  task  in  stratification  is  choosing  criteria  by 
which  to  stratify.  This  involves  the  characteristic  by  which 
to  stratify,  the  number  of  strata,  and  the  boundaries  defining 
the  individual  strata . 


For  a  large-scale  survey  using  stratified  sampling  and 
appropriate  analytic  procedures,  the  major  objective  is  to  di¬ 
vide  a  heterogeneous  population  into  subpopulations  (i.e., 
strata)  that  are  homogeneous.  If  this  is  done,  a  relatively 
small  sample  from  each  stratum  will  provide  a  precise  estimate 
of  the  stratum  mean,  and  appropriate  techniques  for  combining 
the  stratum-mean  estimates  will  yield  a  precise  estimate  for 
the  population  mean. 
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TABLE  XX 


COMPARISON  OP  STRATIFIED  AND  SIMPLE  RANDOM  SAMPLING 


Factor 

Comparison 

Planning  of  Sample 

Stratified  sampling  requires  more  detailed 
planning  and  knowledge  of  underlying  main¬ 
tenance-task  population  than  does  simple 
random  sampling. 

Administration  of 
Sampling  Procedure 

Stratified  sampling  includes  all  adminis¬ 
trative  aspects  of  simple  random  samples 
plus  additional  control  to  meet  specifi¬ 
cation  sample-size  criteria. 

Analysis  of  Data 

Standard  analytical  methods  are  based  on 
simple  random  sampling.  Stratified  ana¬ 
lytical  procedures  for  stratified  samples 
are  relatively  complex  and  may  not  be 
available . 

Sampling  Efficiency 

Stratified  sampling  generally  is  more  ef¬ 
ficient  than  simple  random  sampling  in 
that  variances  of  sample  estimates  are 
lower  than  for  simple  random  samples. 

Sub-hypothe  se  s 

Stratified  sampling  provides  a  means  to 
test  hypotheses  on  different  portions 
of  the  system  with  adequate  control.  Such 
control  Is  not  generally  available  for 
simple'  random  sampling. 

Repre  sentat i vene  s  s 

Stratified  sampling  provides  assurance 
that  sample  observations  from  each  stra¬ 
tum  will  be  observed.  Simple  random  sam¬ 
ples  can  only  provide  such  assurance 
probabill st ically . 

In  maintainability  demonstration,  however,  known  analytic 
techniques  for  stratified  sampling  are  either  inapplicable  or 
too  complex  for  practical  use.  Stratification  is  used  in  demon¬ 
stration  primarily  to  ensure  representativeness;  therefore,  the 
criterion  of  homogeneity  should  be  considered  with  this  in  mind 
If  a  stratum  Includes  a  number  of  tasks,  selection  of  only  one 
or  two  tasks  should  yield  a  representative  sample  of  the  stratum, 
and  across  all  strata  the  sample  selections  should  accurately 
represent  the  total  maintenance-task  population. 
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Therefore,  the  tasks  within  a  stratum  should  require  approxi¬ 
mately  the  same  amount  of  maintenance  time  or  the  same  number  of 
man-hours,  whichever  is  appropriate.  Rigid  adherence  to  this 
precept  should  be  avoided,  however.  Repairing  a  particular  elec¬ 
tronic  assembly  may  take  approximately  the  same  amount  of  time 
as  repairing  a  motor  generator,  but  the  differences  between  the 
two  types  of  actions  would  make  it  unnatural  to  place  them  in 
the  same  stratum.  It  seems  reasonable,  then,  to  require  also 
that  there  be  similarities  among  the  tasks  assigned  to  a  stra¬ 
tum. 

The  detail  to  which  the  maintenance  tasks  are  defined  must 
also  be  considered.  For  example,  a  maintenance  action  may  be 
defined  as  a  single  task,  "replace  Unit  01"  or  as  two  tasks, 
such  as  "replace  Unit  01,  which  has  a  3hcrt"  and  "replace  Unit 
01,  which  has  an  open".  Conceivably,  the  diagnostic  times  for 
these  two  tasks  can  differ,  so  that  they  might  better  be  placed 
in  different  strata.  There  are  practical  limits  on  the  detail 
to  which  tasks  are  defined,  however  —  namely;  the  amount  of  in¬ 
formation  concerning  expected  task  times  and  the  desire  to  limit 
the  number  of  strata  so  that  random- sampling  analytical  tech¬ 
niques  can  be  reasonably  applied. 

There  is  no  single  correct  approach  to  stratifying  the  popu¬ 
lation  of  maintenance -task  times.  The  following  approach  is* be¬ 
lieved  to  be  reasonable  and  practical; 

(1)  First  divide  the  equipment  or  system  by  physical  en¬ 
tities,  such  as  equipments  within  a  system  or  units 
within  an  equipment.  These  first-level  breakdowns 
will  be  called  blocks. 

(2)  For  each  block,  subdivide  to  the  highest  system  level 
at  which  maintenance  will  be  performed.  If  the  block 
is  the  highest  level,  no  further  subdivision  is  neces¬ 
sary.  If  an  equipment  is  under  tes^  and  the  organiza¬ 
tional-maintenance  philosophy  la  unit;  replacement, 
subdivide  to  the  units.  In  3one  cases,  repairs  or 
adjustments  may  be  made  within  a  unit,  but  this  Is 
considered  in  the  next  step.  These  elements  of  one 
subdivision  will  be  called  sub-blocks. 

(3)  For  each  sub-block,  list  the  associated  maintenance 
tasks  and  estimated  maintenance -tack  times  or  man¬ 
hours.5  For  a  sub-block  that  is  an  LRU,  removal  or 
replacement  may  be  the  only  task  listed.  However,  If 
LRU  adjustment  or  some  further  task?  such  as  crystal 
replacement  are  possible.,  they  would  also  be  listed 
as  sub-block  tasks. 


5 It  ie  assumed  that  only  tasks  which  relate  to  apecified  main¬ 
tainability  index  are  listed.  For  example,  noncritical,  frequently 
occurring,  and  easily  corrected  malfunctions/  such  as  light-bulb 
failures,  would  not  normally  be  included  in  the  task  list. 


(4)  Group  together  those  tasks  in  each  sub-bloc.,,  which 
require  essentially  similar  actions  and  will  be  ex¬ 
pected  to  have  similar  maintenance  times:  or  man-hours, 
whichever  index  applies.  The  use  of  historical  data, 
the  prediction  effort  of  the  maintainability-engineer¬ 
ing  group,  and  previous  development  tests  should  be 
used  as  inputs  for /the  time  estimates.  These  groups 
will  then  form  par j;  of  the  initial  set  of  strata . 

(5)  Compile  a  list  of  maintenance  tasks  that  cannot  be 
easily  associated  with  a  physical  entity  —  for  exam¬ 
ple,  repair  of  Interconnecting  cable;  and  tasks  that 
are  net  a  direct  result  of  unreliability  or  degrada¬ 
tion,  such  as  equipment  abuse  and  faulty  installations 

(6)  Combine  similar  tasks  of  step  5  into  strata,  which 

are  then  added  to  the  list  of  initial  strata  of  step  4. 

Thl3  scheme  simply  uses  hardware  characteristics  as  the 
first  approach  towards  stratification  and  then  similarity  of 
maintenance  actions  and  task  times  within  hardware  groups  as 
the  final  criterion  for  stratification.  The  miscellaneous*  tasks 
of  step  5  are  added  in  recognition  of  the  fact  that  not  all  main¬ 
tenance  actions  Involve  the  usual  "black  box"  failures  and  not 
all  actions  are  predictable  simply  from  analysis  of  part  failure 
rates . 


This  initial  set  of  strata  may  have  tc  be  revised  when  the 
actual  tasks  to  be  induced  and  sample-size  requli’ements  are  con¬ 
sidered.  Later  examples  illustrate  this.  At  this  point  it  is 
worthwhile  to  provide  an  illustration' of  this  procedure. 


The  illustration  will  be  concerned  with  a  maintainability- 
demonstration  test  of  an  airborne  dopp~er  radar  equipment  con¬ 
sisting  of  the  following  units; 

Antenna  (AS) 

Receiver/Transmitter  (RT) 

Frequency  Tracker  (FT) 

Radar  Set  Control  (C) 

Drift  Angle  Indicator  (ID) 


The  organizational-maintenance  procedure  for  this  equip¬ 
ment  prescribes  replacing  all  units  except  the  reeelver-trans?- 


mit-ter, 
placed . 


for  which  modularized  assemblies  are  removed  and  re 
These  assemblies  are  as  follows: 


I.F.  -  A 
I.F.  -  B 

Audio  Amplifier  (Amp) 


Modulator  (Mod) 
Transmitter  (Tx) 
Power  Supply  (PS) 


Simple  mechanical  repairs  or  adjustments  on  the  antenna 
and  crystal  replacement  In  the  frequency  trackers  are  also  per¬ 
formed  in  the  aircraft.  The  stratification  procedure  described 
above  le  shown  schematically  in  Table  XXI. 


TABLE  XXI 


EXAMPLE  OP  STEP-BY-STEP  STRATIFICATION 


Step  1 
Blocks 


Step  2 
Sub- blocks 


Step  3 

Sub-block  Step  4  Step  5 

Tasks  and  Block  Strata  Miscellaneous 
mo  c  \r  strata 


Task  Times 


Antenna 


Antenna 


IP  -  A 
IF  -  B 

Receiver/  Amplifier 
Transmitter  Modulator 

Power  Supply 
Transmitter 


|  Frequency 
i  Tracker 


Radar  Set 
Control 


Frequency 

Tracker 


Radar  Set 
Control 


R/R  1.0 
Mech. 

Adjust  0. 5 

R/R  0.3  \ 
R/R  0.3  J 
R/R  0.4  1 
R/R  0.4  > 
R/R  0,4  J 
R/R  0.5 

R/h  0.6 
Replace 
Crystals 
0.5 

R/R  0.c 


A  -  R/R 
A  -  Mech. 
Adjust . 

IF  -  P./R 

Amp,  Mod., 

PS  -  R/R 

Tx  -  R/R 

FT  -  R/R 
I  FT  -  Replace 
|  Crystals 


Drift  Angle  Drift  Angle  R/R  C.5  ID  -  R/R 
Indicator  Indicator  , 


[Miscellan¬ 

eous 


*R/R  -  remove  and  reolace. 


/Repair-  Ca- 
jbling  or  con¬ 
nectors  . 
Adjust  fau.lty 
I instaJ lation 


6. 4 ,2.3  Task  Occurrence  Frequency  and  Selection 

Once  the  initial  set  of  strata  has  been  established,  it  Is 
necessary  to  estimate  the  feauency  of  occurrence  of  tasks  In 


each  stratum ,  For  taske  that  result  from  part  failure,  tha  use 
of  r^rt  failure  rates  such  as  those  presented  In  MIL-KCG3K-217 
may  be  satisfactory.  These  failure  rateB,  however,  primarily 
reflect  catastrophic  piece-part  failures  and  usually  do  not  in- 
elude  such  failure  modes  as  degradation,  part  interactions,  and 
intermittencies . 

Since  it  Is  quite  difficult  to  predict  these  types  of  fail¬ 
ures,  a  simplified  procedure  is  used  whereby  a  failure-rate  pre¬ 
diction  based  on  catastrophic  failures  is  adjusted  to  account 
for  other  types  of  malfunctions.  One  means  for  such  adjustment 
is  to  analyze  unit  complexity. 

Previous  ARINC  Research  analysis®  of  historical  data  on  elec 
tronlc  and  electromechanical  systems  has  indicated  that  the 
operating  failure  rate  of  a  nonredundant  item  can  be  estimated 
by 


where 


A 


A  = 


KC1,35 


A  is  the  estimated  failure  rate 

C  Is  the  complexity  *n  terms  of  equivalent  analog 
active  elements 

K  is  the  average  failure  rate  of  one  active  element 

s 

operating  in  environment  e  (i.e,,  ground,  airborne, 
cr  missile) 

This  type  of  relationship  also  appears  in  the  Navy  Main¬ 
tainability  Engineering  Handbook  -  NAVORD  CD  39223.  Table  XXII 
is  abstracted  from  that  document  to  provide  the  basis  for  deter¬ 
mining  C,  the  number  of  equivalent  analog  active  elements  in  an 
equipment.  The  fact  that  the  exponent  of  the  conplexity  factor 
is  greater  than  one  is  attributed  In  part  to  the  Interaction 
effects  existing  within  the  system,  which  can  cause  noncata- 
strophic  failure  occurrences  suv.h  as  component  degradation  and 
Intermit tency. 


The  above  equation  nan  be  used  in  one  of  two  ways  in  as¬ 
sessing  freq\;er>cy  of  occurrence  of  failure  In  electronic  or 
electromechanical  items: 


(1) 


If  MIL-KDBK-217  or  similar  failure-rate  prediction 
procedures  are  used,  the  total  failure  rate  of  the 
item  can  be  estimated  by  the  equation 


A  = 


^cat 


c°.35 


foG.  T.  Bird,  "On  Reliability  Prediction  in  Satellite  Systems ", 
ARINC  Research  Publication  4225-1-205,  I960  (see  summarizing 
article  in  May  23,  1930  issue  of  Av  at ion  Week). 
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TABLE  XXII 


WEIGHTED  FACTORS  FOR  ESTIMATES*  EQUIVALENT  ANALOG 
COMPLEXITY  OF  ELECTRONIC  SUBSYSTEMS 


AEG  Type 

Function 

Equivalent 

Analog 

AEGs* 

Transistor 

Signal -level  analog  function 

1.0 

Signal -level  digital  function 

0.1 

Power  conversion  and  regulation 

2.0 

Diodes  (Semiconductor) 

Signal -level  analog  function 

0.1 

Signal-level  digital  function 

0.01 

Power  rectification 

1.0 

Electron  Tube 

Signal -level  analog  function 

1.0 

Signal -level  digital  function 

0.1 

Power  conversion  and  regulation 

10.0 

Microwave  Power  Tubes 

Traveling  wave  tubes. 

magnetrors,  klystrons 

100.  C 

Photoelectric  Cell 

Light  sensor  functions 

0.1 

Photo  Multiplier 

Light  amplifier 

10.0 

Solar  Cell 

Power  generation 

0.01 

Relays 

General 

;  .0 

Gyros,  Position 

Inertial  reference 

50.0** 

Gyros,  Rate 

"Rate"  signal 

10.0*« 

Accelerometers 

Acceleration  measurement 

1.0 

Crystals 

Frequency  determination 

1.0 

Thyratror. 

Power  switching 

100.0 

*  See  NAVORP  OP  39223,  Appendix  B  for  source  of  AEG  data. 

**  For  short-duration  missions  (less  than  500  hours). 
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where  is  t3:ie  total  f&llure-rate  estimate  based  on 
catastrophic  occurrences  only. 

(2)  If  a  direct  prediction  of  total  failure  rate  1b  to  be 
obtained  frcm  complexity  ahalyaia.  the  following  equa¬ 
tions  are  used  (X  *»  failure®  per  operate  hour)? 

Ground  Systems:  inX  =  -12.013  +  1.35fnC 

© 

Airborne  Systems:  lnXQ  *  -10,005  +  1 -35inC 

Missile  Systems:  ~  -6.950  1.35/nC 

For  mechanical,  hydraulic,  and  other  nonelectronic  items,  ap¬ 
propriate  prediction  procedures  presented  in  the  literature  will 
have  to  be  used.7  In  some  cases,  direct  failure-rate  predictions 
are  inappropriate,  and  the  probability  of  task  occurrence  in 
terms  of  unreliability  1b  a  better  measure.  Such  probabilities 
can  be  adjusted  to  an  equivalent  average  failure  rate  by  the 
relationship 


where  U(t)  is  the  prooability  that  the  maintenance  task  will  be 
required  after  a  mission  of  t  hours  [U(t)  =  l-R(t)] . 

Prediction-by-function  approaches  are  also  applicable  for 
failure-frequency  estimation  since  they  are  generally  based  on 
field  data  that  Include  all  type3  of  failures.  Some  prediction' 
by-function  equations  include  complexity  as  a  prediction  para¬ 
meter.  The  following  RADC  reports  are  applicable? 

RABC-TDR-63-146  -  "System  Reliability  Prediction  by  Func¬ 
tion"  (Ground  Systems)  -  Federal  Electric  Corporation,  May 

1963,  ad  406191. 


7Several  such  sources  are: 

RADC-TR  68-403,  "Reliability  Prediction  -  Mechanical  Stress/ 
Strength  Interference  (Nonferrous) ",  University  of  Michigan, 
February  1969. 

RADC-TR  68-114,  "Data  Collection  for  Nonelectronic  Reliability 
Handbook",  Hugheo  Aircraft  Corporation,  June  1968. 

"Investigation  of  Reliability  of  Mechanical  Systems",  Lockheed- 
Qeorgla  Company.  October  1965,  AD  475977, 


KADC-THt-63-300  -  "System  Reliability  Prediction  by  Func¬ 
tion"  (Ground  Systems,  2  Vols.),  ARBJC  Research  Corporation 
August  1963,  AD  481191,  481192. 

RADC-TBR  65-27  "System  Reliability  Prediction  By  Function" 
(Ground  Systems  -  Supplementary  Report),  ARINC  Research 
Corporation,  March  1965,  AD  614227. 

RADC-TDR  66-509  -  "Avionics  Reliability  and  Maintainability 
Prediction  by  Function,"  ARINC  Research  Corporation,  Octo¬ 
ber,  1966,  AD  802998. 

Once  the  failure-rate  predictions  are  made,  the  relative 
frequency  of  task  occurrence  is  calculated  in  a  manner  similar 
to  that  indicated  in  Table  XIX  of  Subsection  6. 4. 2,1.  Table 
XXIII  shows  the  computations  for  the  illustrative  system. 

In  the  table,  part-failure-rate  estimates  corrected  by  the 
complexity  factor  are  assumed  to  be  used  for  all  tasks  except 
those  Involving  mechanical  faults.  For  the  latter,  the  failure 
rate  shown  is  assumed  to  be  based  on  estimates  of  occurrence 
probability.  Dividing  the  total  failure  rate  of  830  into  the 
individual  maintenance-task  strata  rates  yields  the  relative 
frequencies  of  occurrence  shown  in  the  table. 

Several  strata  are  then  regrouped  to  yield  at  least  a  five 
percent  frequency  of  occurrence .  This  regrouping  is  done  to  mini 
mlse  the  number  of  strata  with  small  frequencies  of  occurrence, 
especially  those  which  would  lead  to  a  required  sample  size  of 
less  than  one  for  stratified  sampling  It  is  not  necessary  to 
regroup  if  simple  random  sampling  is  to  be  used,  in  which  case 
the  relative  frequencies  of  occurrence  are  used  to  determine 
task  selection.  If,  for  example,  the  required  total  sample  size 
is  60,  then  60  numbers  she.’  Id  be  drawn  from  a  random-number 
table.  If  a  random  number  is  between  0  and  0.16,  a  maintenance 
task  involving  the  antenna  is  to  be  simulated .  If  the  random 
number  is  between  0.16  and  0.27,  e  fault  resulting  in  the  re¬ 
moval  and  replacement  of  an  IF  module  is  induced,  etc. 

For  a  stratified  sample,  the  numbers  In  the  last  column 
of  the  table  are  the  sample-size  requirements.  Thus,  for  a 
sample  size  of  60 }  10  faults  are  to  be  induced  involving  the 
antenna,  7  faults  are  to  be  induced  that  will  result  in  the  re¬ 
moval  and  replacement  of  one  of  the  IF  modules,  etc . 

1 

x  The  actual  faults  to  be  induced  require  further  analysis. 
For  the  antenna,  it  is  seen  from  the  relative-frequency-of -occur 
rence  column,  that  removal  and  replacement  occurs  almost  four 
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TABLE  XXIII 

CALCULATIONS  OP  RELATIVE  FREQUENCE  OP  OCCURRENCE 
AND  SAJ-iPLE  SIZE  FOR  RADAR  SOPPIER  EQUIPMENT 


Maintenance  Task 
Strata  ^ 


Failure  Quantity  Total  Relative  Frequency  - 
Rate  of  Failure  of  Occurrence,  P^  Sllett 


Items  Rate# 


{ Percent ) 


Antenna  R/R 

Antenna-Mech/Adjust . 

IF-R/R 

Amp  -  R/R  i 
Mod  -  R/R  V 
P.S.  -  R/R  * 

Tx  -  R/R 

FT  -  R/R 

FT  Replace  Crystal 
C  -  R/R 
ID  -  R/R 

Repair  Cabling  or 
Connector 

Adjust  Faulty 
Installation 


Total 


105 

30#* 

90 


°-“n  0.25f 

0.036J 

0.109  -  0.11 


0.036 


|0.05t 


0.012  1 

0.482  -  0.48 

0.096  -  0.10 


35  C.042 ' 
10  0.012 
10**  0.012' 


30**  0.036 


1.000 


}  0  05t 


*A11  units  have  100$  duty  cycle.  Therefore,  a  duty-cycle  column  is  not  shown. 
♦•Obtained  from  estimate  of  probability  of  occurrence,  using  the  relationship 

?v  =  =  mission  time  . 

t  Regrouped  to  increaseall  frequencies  of  occurrence  to  at  least  0.C5. 

ft  The  sample  sizes  shown  apply. only  to  stratified  sampling.  For  simple  random 
sampling,  the  relative  frequencies  of  occurrence  are  used. 


times  as  often  as  the  mechanical  adjustment .  If  the  mainten¬ 
ance  task  Involves  the  antenna,  the  probability  that  it  is  a 
removal  and  replacrment  action  is 


«T.T5?'+2Lw"0-78 

Therefore,  a  representative  random  sample  for  the  antenna  tasks 
can  be  obtained  by  selecting  a  random  number  between  0  and  1 .  If 
It  Is  less  than  or  equal  to  O.78,  Induce  a  fault  that  will  re¬ 
sult  in  a  removal  and  replacement ,  If  the  random  number  is 
greater  than  O.78,  induce  a  fault  requiring  on-board  mechanical 
adjustment  or  repair. 

Within  these  tasks,  there  will  also  be  a  choice  of  the  fault 
to  be  simulated.  This  may  involve,  for  example,  the  selection  of 
an  assembly  or  part  and  its  mode  of  failure,  and  here  the  con¬ 
sideration  of  symptom  indication  may  be  important,  especially  for 
removal-replacement  actions  (see  Subsection  6. 4. 2. 4  for  a  sampling 
approach  based  on  symptoms). 

In  the  same  fashion  as  discussed  above  with  regard  to  the 
choice  of  antenna  task,  failure-mode  or  symptom  probabilities 
should  be  analyzed  to  obtain  their  relative  frequency  of  occur¬ 
rence  within  a  task.  A  random- sampling  procedure  is  then  ap¬ 
plied  in  accordance  with  these  relative  probabilities  to  deter¬ 
mine  which  faults  should  be  Induced. 

For  example,  consider  the  remove-and-replace  task  involving 
the  frequency  tracker.  From  Table  XXIII,  28  tasks  are  to  be 
simulated  for  a  proportional  stratified  sample.  A  failure-mcde- 
and-effects  analysis  indicates  that  there  are  five  major  failure 
modes  that  require  a  remove-and-replace  action  and  that  are 
detectable  at  the  unit  level.  These  modes,  their  effects,  and 
relative  frequencies  of  occurrence  are  as  follows: 


Failure 

Mode 

Effect 

Relative  Frequency 
of  Occurrence  ($) 

Cumulative 
Range  (%) 

1 

Inopers  tive 

30 

0  - 

29.95 

2 

Will  not  lock  on 

20 

30.00  - 

49.99 

3 

Breaks  lock 

.20 

50.00  - 

4 

Drifts 

15 

84.99 

5 

Erratic 

15 

85.00  - 

100. 00 

To  determine  which  faults  to  induce  for  simulating  the  28 
remove-and-replace  tasks,  28  four-digit  numbers  are  selected  from 
a  random-number  table.  If  the  number  selected  is  between  0  and 
2999 j  then  a  fault  or  malfunction  that  makes  the  frequency  tracker 
inoperative  must  be  induced.  A  random  number  between  3000  and 
4999  would  indicate  that  a  fault  resulting  in  inability  to  lock- 
on  effect  is  to  be  simulated,  etc. 

Within  any  failure  mode,  a  choice  would  then  have  to  be  made 
concorrlng  the  specific  means  of  fault  inducement  (e.g.,  which 
lead  to  disconnect  or  which  part  to  replace  with  a  known  faulty 
part) .  If,  with  respect  to  the  maintenance  action,  there  Is  no 
discernible  difference,  the  simplest  means  can  toe  used.  If,  how¬ 
ever,  the  fault  selection  can  affect  maintenance  time  (e.g,,  dis¬ 
connecting  one  lead  nu.y  cause  secondary  symptoms,  while  discon¬ 
necting  another  will  net) ,  then,  again,  a  random-selection  cri¬ 
terion  is  advisable. 

To  minimise  the  biasing  problem?  e  to  task  rehearsals 
and  the  occasional  problem  of  not  be.  able  physically  to  in¬ 
duce  the  required  fault,  it  is  prudent  to  select  a  much  larger 
number  of  possible  tasks  than  required.  For  example,  if  the 
required  sample  size  is  60,  a  total  of  120  taBkB  may  be  selected 
initially  and  procedures  for  fault  inducement  established. 
the  test  Is  actually  run,  half  c f  these  prepared  tasks  are  then 
selected  for  actual  observation. 

6 . 4 . 2 . 4  Stratification  and  Task  Selection  Based  on  the 
Symptom  Matrix 

Another  approach  to  developing  strata  and  selecting  tasks 
is  to  use  the  symptom-matrix  formulation  presented  in  Volume  I 
of  this  study.  The  symptom  matrix  of  concern  here  is  one  that 
relates  unit  malfunctions,  associated  symptoms,  and  occurrence 
probabilities . 

If  symptoms  are  considered  rather  than  maintenance  tasks 
only,  recognition  is  given  to  the  fact  chat  for  many  equipments, 
especially  those  with  easily  replaceable  units,  a  large  portion 
of  the  corrective  maintenance  action  involves  faul  location; 
therefore,  the  use  of  symptom  information  rather  than  task  in¬ 
formation  might  be  a  better  basis  for  stratification. 

The  stratification  can  be  accomplished  in  one  of  three  basic 

ways: 

(1)  Stratify  by  symptoms,  and  sample  unit  malfunctions 
within  symptoms 

(2)  Stratify  by  units,  and  sample  symptoms  within  unit 
malfunctions 
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(3)  Stratify  by  symptom/unit  malfunction  combinations 

To  illustrate  these  three  approaches,  the  symptom  matrix 
developed  for  the  APN-147  Doppler  Radar  Set  monitored  in  this 

study  is  shown  in  Table  XXIV. 9  The  entries  in  the  cell  repres¬ 
enting  the  i*'*1'  unit  and  the  j^h  symptom  are  described  as  follows 


1th  Unit 


where 

h 

X.  ,  =  the  failure  rate  of  elements  in  the  I','~  unit 
whose  malfunction  will  result  in  the  J  symptom 

f(S  |U, )  «  the  probability  that  the  jth  symptom  will 
J  x  th 

occur  given  the  i  unit  malfunctions 

P(U.  1  S  )  =  the  probability  that  the  ith  uric  has  mal- 
functioned  given  the  j '**  symptom  appears 

P(UlSj)  =  the  probability  of  the  joint  occurrence  of 
an  1th  unit  malfunction  and  jth  symptom 

One  difficulty  in  using  this  approach  is  to  develop  the 
estimates  since  this  requires  fairly  detailed  analysis  of' 

the  system  desjgn.  Volume  I  presents  a  discussion  of  this  es-^ 
tlmating  process. 

The  following  relationships  exist  under  the  assumption  that 
concurrent  failures  do  not  occur: 

P(S  IU,  )  ~  =  -ill 

J  f*ik  \ 


®Only  major  units  are  shown.  Such  Items  as  mounting  trays  and 
cabling,  which  can  and  do  sometimes  caxise  failure,  are  not  In¬ 
cluded  in  the  example. 


.th  . 
j  symptom 


p(ui1 Sj ) 

P(Sj|  u±) 
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11  Entry 


A, 


^hj  "S 

p (H  )  „  " _  _  ,_A 

1  ^  >T 


hk  ** 


^  ik  }'U. 
P (IT.  U  -£ —  =  --J- 

1  ^iAhk  *T 

hr. 


where 


»T 


=  the  failure  rate  of  the  1th  unit 

=  the  failure  rate  of  all  items  whose  malfunction 

th 

will  produce  the  j  symptom 
=  the  total  equipment  failure  rate 


For  simplicity,  assume  that  n  =  100  maintenance  actions 
are  to  he  induced  by  stratified  sampling.  Under  the  three 
methods,  the  sampling  described  below  would  take  place.9 


(l)  Symptom  Stratification.  The  number  of  tasks  in  which 
the  j  1  symptom  is  induced  is  P(Sj}*n.  Hence  the  tdsk 
sample  sizes  by  symptom  are  as  follows: 


Symptom 

Sample  Size 

Symptom 

Sample  Size 

1 

8 

1  and  4 

76 

2 

4 

1,  4,  and  7 

0 

4 

6 

2  and  5 

r>, 

C 

7 

2 

3  and  6 

2 

sFor  simple  ’andom  sampling,  the  procedure  is  analagoue  to 
that  discussed  for  maintenance-taak  stratification. 
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Within  each  symptom,  malfunctions  are  introduced  ran¬ 
domly  into  che  units  according  to  the  probabilities 
P{Ui'£j'*  Thus  for  generating  symptom  S^,  there  should 

be  a  O.63  probability  of  inducing  the  symptom-generat¬ 
ing  fault,  in  the  frequency  tracker,  a  0.20  probability 
for  inducing  it  in  the  antenna,  a  0.03  probability  for 
inducing  it  in  the  indicator,  and  a  0.13  probability 
fir  Inducing  it  In  the  control  unit. 

(2)  Unit  Stratification.  The  number  of  tasks  in  which 

.  -  ...  —  - -  -  —  — — * - 

the  i  unit  is  induced  is  P(U^)*n.  Hence,  the  task 

sample  sizes  by  unit-failure  inducements  are  as 
follows: 


Unit 


Sample  Size 


Within  each  unit,  malfunctions  are  introduced  ran¬ 
domly  according  to  the  probabilities  P(SjU^).  Thus 

for  the  antenna  there  should  be  a  0.17  probability  of 
inducing  a  fault  that  will  generate  S^,  a  O.33  proba¬ 
bility  for  S2,  and  a  0.50  probab  lity  for  the  and 
symptom  combination. 

(3)  Symptom/Unit  Malfunction  Stratification.  -For  this 
case,  each  unique  symptom/ unit  malfunction  cornel,  n- 
tjon  represents  a  stratum.  All  the  strata  are  thus 
combinations  with  non-zero  entries  for  P(U1Sj),  and 

the  sample  sizes  are  P(b^Sj)'n.  Thus  22  samples  are 

to  be  selected  with  symptoms.  1  and  4  with  an  Rf  mal¬ 
function,  5  samples  aie  to  be  selected  with  symptom  1 
with  a  malfunction  in  the  frequency  tracker,  etc. 

Tne  choice  between  the  three  basic  approacnts  described 
above  depends  on  the  expected  comparative  representativeness 
and  on  sampling  difficulties .  The  third  method  is  probably 
leas+  nractical  since  it  will  genera j.] y  entail  too  fine  a  break 
down,  requiring  regrouplrg  of  some  combiner  ions .  Method  2  Is 
probably  preferable  to  Method  i  because  Method  1  may  place  too 
much  emphasis  on  the  symptom  aspects  of  maintenance. 


Method  2  is  actually  the  same  initial  approach  as  the  task- 
sampling  procedure  described  earl.er.  ^he  difference  is  that 
1  or  this  method  the  actual  faults  to  be  introduced  within  a  unit 
are  governed  by  symptom  occurrence  rather  than  failure  occurrence. 

The  symptom  approach  can  be  recommended  over  the  task- 
sampling  approach  in  the  following  circumstances: 

Fault  location  and,  to  a  lesser  extert,  checkout  time 
are  expected  to  account  for  the  most  significant  por¬ 
tion  of  the  corrective  maintenance  action. 

The  necessary  information  and  resources  (personnel,  time, 
money)  for  developing  the  symptom  matrix  are  available. 
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statistic;,  l  maintain abi  li  ty- demonstration 


PLANS 


7 . 1  GENERA  L 

This  section  reviews  the  maintainability-demonstration  test 
methods  of  MTL-STP-471  and  then  presents  a  number  of  ether  tests 
applicable  to  maint a  inability  demonstration.  Each  of  the  plans  1 
through  4  of  MIL-3TD-471,  which  are  the  most  frequently  applied 
procedures,  has  one  or  more  analogs  in  the  set  of  alternative 
plans  presented  herein. 

Pour  different  categories  of  alternatives  are  presented: 

(1)  Fixed-samplc-size  tests,  lognormal  distribution 

(2)  Sequential  tests,  lognormal  distribution 

(.3)  Nonparametric  tests 

(4)  Bayesian  tests 

For  convenience,  the  following  standardized  format  is  used 
to  describe  the  alternative  plans  for  the  non- Bayesian  tests: 

•  General  Description  of  Test 

•  Underlying  Assumptions 

•  Hypothesis 
Sample  Size 

•  Decision  Procedure 

■  •  Discussion 

Guidelines  for  selecting  among  the  alternatives  are  also 
presented . 

7.2  MIL-STDJ+71  PLANS 
7.2.1  General 


The  six  test  methods  of  MIL-STD-471  are  reviewed  in  this 
subsection.  These  test  methods  include  situations  covering  various 
types  of  maintainability  specifications,  test  procedures,  and 
underlying  assumptions.  Table  XXV  summarizes  the  various  forms  of 
maintainability  parameters  that  are  tested  by  five  of  the  six 
MIL-STD  test  methods  under  consideration.  Test  method  5  is  not 
actually  a  decision  test  but  an  approach  to  estimating  the  per¬ 
centage  of  maintenance  taslcs  between  the  observed  sample  extremes; 
therefore,  it  is  not  listed  in  the  table,  but  it  is  reviewed. 
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Test 

Method 


1 

2 

3 

4 
6 


TABLE  XXV 

SUMMARY  OF  MIL- STD-4 71  SPECIFIED  PARAMETERS 
M 


M 


ct 


X 

X 


pt 


M 


X 


‘'max  ct 


X 

X 


M 


max  pt 


M 


ct 


JL 


X 

X 


M 


pt 


Hot  “ 

H  .  = 

Pt 

ffl  = 


M. 


M, 


max  ct 

max  pt 

Mct  " 

M,  = 

Pt 


Mean  corrective-maintenance  downtime 

Mean  preventive-maintenance  downtime 

Mean  maintenance  downtime  consisting  oi  cor¬ 
rective  and  preventive  in  the  same  time  period 

1  percentile  of  corrective-maintenance  down¬ 
time 

p  percentile  of  preventive-maintenance  down¬ 
time 

Median  corrective-maintenance  downtime 
Median  preventive-maintenance  downtime 


The  six  methods  are  reviewed  below  with  respect  to  type  of 
test,  conditions  of  use,  sample  sizes,  and  accept/reject  criteria. 
The  reader  is  advised  to  consult  the  standard  for  the  detailed 
test  procedures. 


7  2  Test  Method  1 

Type  of  Test.  Two  sequential  tests  are  performed  —  one  for 
and  one  for  Both  tests  must  be  passed  for  an  accept 

decision.  The  producer’s  and  consumer’s  risks  are  held  to  a  maxi¬ 
mum  of  16  percent. 

Conditions  of  Use.  and  must  both  be  specified. 

A  lognormal  distribution  of  corrective-maintenance  times  is 
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assumed ,  although  it  is  claimed  that  test  risks  are  only  slightly 
changed  if  the  actual  distribution  is  exponential  or  normal.  The 
specified  M  must  be  greater  than  10  minutes  and  less  than  IOC 

minutes,  and  the  ratio  M  „  ./M  .  must  be  less  than  3*  M  . 

max  ct  ct  max  ct 

can  be  either  the  90th  or  95th  percentile. 

i 

Sample  Size.  The  sequential  tests  are  truncated  at  100 
maintenance  actions .  Fewer  than  that  number  will  generally  be 
required  before  a  decision  is  reached. 

Accept/Reject  Criteria.  The  number  of  maintenance  times  less 

than  and  greater  than  the  specified  M  .  and  M  .  are  recorded 

ct  max  ct 

and  compared  with  tabulated  values  of  the  accept  and  reject  num¬ 
bers  for  the  two  sequential  bests.  A  decision  rule  for  cases  in 
which  no  decision  nas  been  made  after  100  observed  actions  is 
also  provided . 

Discussion.  The  Standard  calls  for  stratified  samDling. 
which  may  seriously  affect  the  risks  of  the  sequential  plan  un¬ 
less  the  order  of  the  sampling  is  strictly  random.  It  is  empha¬ 
sized  that  the  specified  maintainability  requirements  and 

Mmax  ct-*  arc  the  unacceP^at)le  levels  —  that  is,  equipment  which 

exactly  meets  these  levels  will  have  a  low  probability  of  passing 
the  test.  It  is  also  emphasized  that  the  procedure  is  based  on 
converting  the  mean  and  percentile  specifications  to  equivalent 
e pec if i cations  of  binomial  parameters,  leading  to  the  use  of 
sequential  tests  when  the  binomial  distribution  applies. 

7.2.3  Test  Method  2 


Type  of  Test.  Fixed-sample  test  employing  the  central- limit 
theorem  for  sample  statistics  of  WQt,  an<3  Mmax  ct‘  Only 

consumer's  risks  are  considered;  the  value  of  0  is  contractually 
determined . 

Conditions  of  Use.  To  demonstrate  M _ a  lognormal 

""  ™  ^  l _  max  Ct 

repair- time  distribution  must  be  assumed.  When  both  preventlve- 

and  corrective-maintenance  indexes  are  specified,  the  proportion 

of  each  type  of  maintenance  during  a  representative  operational 

period  must  be  estimated. 

Sample  Size.  A  minimum  of  50  corrective-maintenance  tasks 
and  50  preventive-maintenance  tasks  (if  M .  is  specified)  must  be 
observed.  p 

Accept/Reject  Criteria.  Equations  are  given  for  estimating 

M  . ,  M  ,  and  M  . .  Accept/reject  values  for  each  of  these 
Cu  ot  max  c  Xf 
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vaiue 


Indices  :r.rr.  cnt-’ri  frc>”.  me  formulas  presented. 

tne  Z2.7.L  le  sizes,  cr.e  specified  £  risks, 
-  value  -f  the  pt’’  percentile  of  interest. 

*7  TTf  %T 

‘ct  ’  pt  ‘ 

ing  critical  values,  the  equipment,  passes  the  test. 


a  f  evct  i 

‘"'max  c  t  ’ 
estimate 


and  M  „  ,  are  less  than  the 

max  ct 


They  are 

and,  for 
If  the 

correspond- 


Dlscusslon.  Notice  1  to  MIL-STD-471,  dated  9  April  1968, 

corrected  an  erroneous  statement  that  the  risk  probability  was 

related  to  the  producer’s  risk.  The  test  for  M  .  simply 

nicijc  ct 

compares  the  estimated  Mmax  ^  to  that  specified;  therefore,  the 

risk  for  this  portion  of  the  test  is  not  as  stated  in  the  standard 
but  i3  approximately  50  percent. 


7.2.4  Test  Method  3 

Type  of  Test.  Fixed-sample  test,  assuming;  a  lognormal  dis¬ 
tribution  of  repair  times . 

Conditions  of  Use.  The  specified  value  of  5l  .  ,  Bay  MQ.  the 

median  corrective-maintenance  time,  is  calculated  from  an  equation 
given  in  the  standard  based  on  an  assumed  standard  deviation  of 
O.55  and  a  known  value  of  say  Mp  fox»  which  only  a  5-percent 

acceptance  probability  (P  risk)  is  desired.  The  accept/reject 
criterion  is  based  on  a  producer’s  risk  of.  5  percent. 

Sample  Size.  A  sample  size  of  20  maintenance  actions  is 
required.  “ 

Accept/Reject  Criteria.  Equations  are  presented  for  obtain¬ 
ing  the  statistics  necessary  to  test  for  acceptable  median  repair 
times.  The  standard  "t"  test  equation  (using  logarithms  of  repair 
time)  is  given  to  ensure  that  tqgre  1b  a  95-percent  chance  of 
accepting  equipments  with  »  MQ  and  only  a  5-percent  chance  of 

accepting  equipments  with  «  Mp 

Discussion.  The  description  and  notation  of  the  test  method 
in  the  standard  can  be  somewhat  confusing.  Actually,  the  basic 
requirement  that  is  assumed  to  exist  is  the  unacceptable  value  of 
the  median,  which  is  denoted  by  ERT^.^  This  value  corresponds 

to  a  P  risk  of  5  percent  (the  Standard  erroneously  states  10  per¬ 
cent).  The  sample  size  of  20  is  fixed  and,  assuming  J  (log-base 
10)  =  0.55,  the  value  of  the  median  that  corresponds  to  a  5- 
percent  producer's  risk  (denoted  by  ERT.-.,  in  the  standard)  is 

given  by  the  equation  ERT  =0.37  ERT  .  It  is  emphasized 

that  ERTSpec  Is  completely  determined  by  the  sample  size  of  20, 
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the  assumed  o,  and  the  known  ERT  ,  and  is  not  based  on  prior 

max 

consideration  of "system  or  logistic  requirements.  Therefore,  it 

is  misleading  to  ascribe  the  word  "specified"  to  ERT  .  The 

spec 

better  way  to  evaluate  test  applicability  is  the  ERT  specifi¬ 
cation  and  corresponding  5  risk.  max 

7.2.5  Test  Method  4 

Type  of  Test.  Nonparametric,  test  for  proportion  to  demon- 

strate  achievement  cf  specified  M  .  or  “M  .  and  M  ,  or  M 

^  ct  pt  max  ct  max  pt 

Conditions  of  Use.  No  underlying  distribution  of  maintenance 

timer  is  assumed”!  ‘Both  a  median  and  maximum  time  must  be  soeci- 

fied.  Test  criteria  are  given  separately  for  the  median  (m)  and 

for  the  95th  percentile  (Mmo„).  The  individual  median  and 

m£LX  max 

tests  are  made  at  either  the  75-percent  or  90-percent  confidence 

level. 

Sample  Size.  The  sample  size  is  50  each  for  corrective  or 
preventive  maintenance  tasks. 

Accept/Reject  Criteria.  For  both  the  median  and  teste, 

critical  values  are  provided  for  the  number  of  maintenance  actions 
greater  than  specified  values  for  a  sample  size  of  50  and  for  the 
75-percent  and  90-percent  confidence  levels.  An  accept  decision 
is  made  only  if  both  the  median  and  tests  are  passed. 

Discussion.  The  individual  median  and  tests  are  equiva¬ 

lent  to  tests  on  a  binomial  parameter.  It  is  important  to  note 
that  the  specified  parameters  represent  unacceptable  maintain¬ 
ability  levels;  e.g.,  the  90-percent-confidence-level  teat  of 

v  V 

is  such  that  if  the  median  corrective-maintenance  time  is  equal 
to  tnat  specified,  there  is  only  a  10-percent  ohance  of  passing 
the  test.  Thus  this  test  is  based  on  specified  (3  risks  (1-con- 
fidence  level)  corresponding  to  the  specified  maintainability 
level  for  median  or  95th  percentile.  The  Beta  risk  corresponding 
to  an  equipment  accept  decision  (both  individual  tests  are  passed) 
is  approximately  1.5  percent  for  th  .  90-percent-confidence-level 
tests  and  9  percent  for  the  75-percent-confidence-level  tests. 

7.2.6  Test  Method  5 


Type  of  Test.  A  nonpararetric  procedure  for  obtaining  a 
confidence- interval  estimate  of  the  proportion  of  maintenance- 
task  times  in  the  population  that  will  be  included  between  the 
observed  sample  extremes.  Tolerance  interval  is  the  usual  termi¬ 
nology  for  such  an  estimate. 
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Conditions  of  Use.  No  underlying  distribution  of  maintenance 
times  is  assumed. 

Sample  Size.  MIL-STD-471  presents  the  required  sample  size 
for  thrte  confidence  levels  (50,  95-  and  99)  and  three  population 
percentages  (90,  95*  and  99).  For  cample,  the  standard  shows 
that  a  sample  size  of  47  is  requii  ,o  be  95-percent  confident 
that  90  percent  of  the  population’ ,  maintenance-task  times  vrill 
be  contained  within  the  observed  sample  extremes. 

Accept/Reject  Criteria.  No  such  criteria  exist  for  this 
test.  Tne  test  results  provide  a  measure  of  spread,  and  this 
measure  may  be  used  to  determine  acceptability. 

Discussion.  Since  this  procedure  is  basically  one  of  esti- 
mation  and  since  it  i3  not  geared  to  a  particular  demonstration 
requirement,  it  should  be  used  more  for  informative  purposes  than 
for  decision  purposes. 

7.2.7  Test  Method  6 


Type  of  Test.  Nonstatistical-type  test, 
are  compared  with  the  specified  values. 


Sample  statistics 


Conditions  of  Use.  Values  of  or  ^pax  pt*  or  both,  must 

be  specified.  The  percentile  point  defining  the  latter  may  be  any 
value.  The  frequency  of  occurrence  for  each  type  of  preventive- 
maintenance  task  must  be  estimated. 


Sample  Size.  Not  specified,  a 11  preventive-maintenance 
tasks  a!re  to  be  performed . 

Accept/Reject  Criteria.  An  average  value  for  is  esti¬ 
mated  from  the  data  on  the  baBis  of  actual  maintenance  times 
weighted  by  frequency  of  occurrence.  This  value  is  then  compared 
with  the  specified  to  determine  acceptance  or  rejection.  To 

test  for  pj.,  the  observed  preventive-maintenance  times  are 

ordered.  The  observed  pth  percentile  time  is  compared  with  the 
specified  to  determine  conformance. 

Discussion.  This  method  provides  no  risk  control  for  either 
producer  or  consumer  and  is  therefore  limited  in  its  application. 
Such  a  procedure  is  better  suited  to  obtaining  information  than 
to  decision- making . 
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7.3  ALTERNATIVE  NON- BAYES IAN  TESTS 

7.3.1  General 


This  subsection  is  devoted  to  the  presentation  of  various 
alternative  tescs  whicn,  to  varying  degrees,  parallel  the  MIL- 
STD-^71  plans  but  offer  greater  flexibility  in  risk  assignment, 
test  parameters,  hypotheses,  and  form  of  testing.  Discussion  of 
Bayesian  tests  is  deferred  until  Subsection  J.j.  The  tests  pre¬ 
sented  are  standard  statistical  tests  or  adaptations  thereof, 
however,  little  or  no  experience  with  their  application  to  main¬ 
tainability  demonstration  has  been  accumulated.  They  a^e  be¬ 
lieved  to  be  practical  for  this  purpose,  but  final  judgment 
must  await  more  extensive  use. 

7.3.2  Comments  or.  Sampling  Procedure  and  Sample  Size 

7-3. 2.1  Sampling  Procedure 

A  general  comment  concerning  the  type  of  sampling  is  in 
older.  All  tests  (both  MIL-STD-471  and  the  alternative  plans) 
employ  analytical  procedures  that  are  based  on  the  assumption  of 
simple  random  sampling.  If  proportional  stratified  sampling  is 
used,  large  sample  sizes  and  a  relatively  small  number  of  strata 
are  required  to  ensure  validity.  In  most  cases,  however,  the 
assumption  of  simple  random  sampling  is  conservative  in  the  sense 
that  the  sample  size  based  on  this  assumption  is  larger  them  would 
be  required  if  emalytical  procedures  were  based  on  stratified 
sampling  errors.  For  sequential  tests,  it  is  recommended  that 
only  simple  random  sampling  be  performed. 

7 . 3 . 2 . 2  Sample  Size 

Tests  for  whi^h  only  a  single  risk  (a  or  p)  is  specified 
do  not  generally  have  associated  sanple-size  requirements.  Since 
the  risk  of  a  wrong'  decision  generally  decreases  with  increasing 
sample  size,  the  larger  the  value  of  the  sample  size,  the  greater 
the  assurance  of  a  correct  decision. 

As  a  very  general  rule.  It  would  be  appropriate  to  specify 
a  minimum  sample  size  on  the  order  of  25  to  30  even  for  cases 
in  which  a  sample- size  equation  Indicates  that  a  lower  value 
would  be  acceptable.  For  very  complex  systems  involving  hundreds 
of  different  types  of  maintenance  actions,  it  will  be  desirable 
to  specify  a  larger  sample  size  to  provide  reasonable  assurance 
of  representativeness. 

f 

7o.3  Lognormal  Distribution 

Since  the  lognormal  distribution  has  often  been  an  adequate 
representation  of  various  repair- type  distributions,  many  of  the 
test  procedures  are  based  on  a  lognormal-distribution  assumption 


concerning  the  maintenance  characteristic  of  interest.  The 
reader  is  advised  to  review  the  characteristics  of  the  lognormal 
distribution  presented  in  Appendix  A. 

7 . 3  •  Notation 

The  following  notation  is  used: 

X  =  the  maintenance-characteristic  random  variable 
p  =  E(x),  the  mean  value  of  X 
Xp  =  the  (l-p)^h  percentile  of  X 

/v 

M  =  XQ  50*  median  of  X 
d2  =  E(X-p)2,  the  variance  of  X 
=  E(ihiX),  the  mean  value  of  inX 
a2  =  E(i!nX-0)2,  the  variance  of  inX 


2  =  the  standardized  normal  deviate  exceeded  with  proba- 

no 


bility  p 


3  i  • e  •  3  J 


-zV2  . 

az  *  p 


For  the  lognormal  distribution,  the  density  is 


tw  -  -7=-;  e  20 

/?7T  CX 


K?  (fnX  -  0)2 


,  0  <  X  <  » 


o 

where  0  =  E[i5n  X]  and  0  =  variance  [in(X)] 


Then  for  the  lognormal  distribution. 


[i  =  e 


e  +  c2.'2 


xp  '  e 


0  +  Z  n 


m  =  e 

d2  .  ea>  +  °2  (go2.^ 


7.3.5  Prior  Estimation  of  0  ^ 

Several  of  the  tests  (1,  2,  ?J}  7,  and  the  Bayesian  test)  re¬ 
quire  that  an  estimate  of  a2  be  used  for  determining  specified 
values,  evaluating  the  sample  size,  or  developing  the  decision 
criterion.  To  aid  in  obtaining  such  estimates,  the  corrective- 
maintenance  data  collected  for  this  study  were  analyzed. 


lable  XXVI  presents  the  observed  a?-  values  for  the  natural 
logarithm  of  active-corrective-maintenance  time  (c^gnkRT)  and 
maintenance  man-hours  per  active  corrective  maintenance  action 
( o2inMH ) .  Time  spent  on  administrative  or  logistic  activities 
is  not  included,  since  these  time  elements  would  not  normally  he 
considered  in  a  maintainability  demonstration.  Variances  are 
shown  for  two  cases:  no-trouble- found  actions  included,  and  no¬ 
trouble-found  actions  excluded.  The  latter  is  generally  more 
pertinent  for  demonstration  in  which  faults  are  to  be  lrduced, 
while  the  former  may  be  more  applicable  for  natural- failure  tests. 

The  sample  sizer  from  which  the  variances  were  calculated 
are  shown  so  that  a  weigh' ed  average  of  several  variance  values 
can  be  used  if  it  is  determined  that  such  an  average  would  b„  best 
for  estimating  the  /ariance  of  the  ec:u  ip  men'1'  under  consideration.  10 

In  addition,  multiple-linear-regr^ssion  analysis  was  em¬ 
ployed  to  develop  prediction  equations  cf  the  form 

a2  =  b0  +  by,  +  b„x2  +....+  brxr 

where 

aO  O  .  N 

0  is  the  predicted  value  of  o  (the  dependent  variable) 

JL . 

is  the  i^  prediction  parameter  (independent  variable) 

bQ,bi,...,br  are  computed  regression  coefficients 

The  least-squares  method  was  applied  through  a  computerized 
procedure  to  obta.'  a  the  regression  coefficients,  as  well,  as  other 
regression  and  correlation  measures  such  as  the  standard  error  of 
estimate,  simple,  partial,  and  multiple  correlation  coefficients, 
and  t  and  F  statistics  for  significance  tests. 

Significance  tests  were  used  to  determine  the  statistical 
significance  of  each  parameter  at  the  nominal  20-perc^nt  signi¬ 
ficance  level.  For  efficiency,  a  step-wise  procedure  was  used. 


10In  FIL-STT-471,  Test  Method  3,  a  value  of  a,  „  =  0.55  is  re- 

lcg10x 

commended  for  use.  On  converting  fo  base-10  logarithms,  the 
average  standard  deviation  of  the  21  eq'uipments  in  Table  XXVI  for 
no- trouble -found  actions  Included  Is  approximately  0.38.  Since 
the  value  In  the  Standard  war,  developed  from  data  obtained  a 
number  of  years  ago.  If  the  two  estimates  are  representative  of 
the  same  populations,  a  desirable  trend  towards  reduced  variation 
Is  evident.  It  can  be  conjectured  that  this  variance  reduction 
is  due  In  part  to  greater  emphasis  on  maintainability,  leading  to 
such  features  as  automatic  test  equipment,  computerized  trouble¬ 
shooting,  and  modular  or  unit  replacement. 
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TAELE  XXVI 


OBS.  -  /ED  VARIANCES  —  AIR  FORCE  AVIONIC 
AND  GROUND  EQUIPMENT 


Equipment- 

No-Trouble-Found  Actions  i 

Included 

No-Trouble-Fcuna  Actions 
Excluded 

Number  of 
Observations 

'2 

linART 

2 

inMH 

Number  of 
Obrervations 

2 

°ZnART 

r 

°|RME 

Avionic  Equipment 

APQ- 110-Terrain  Rader 

25 

0.584 

0.691 

16 

0.629 

0.681 

APN-I67 -Radar  Altimeter 

5 

0.588 

1.388 

<5 

— 

— 

AJQ-20-Ineruial  Bomb  Nav. 

42 

0.37? 

0.590 

22 

0.233 

APQ-113-Fine  Control  Radar 

16 

0.814 

0.808 

6 

1.593 

ARC-109  UHF  Transceiver 

11 

0.430 

0.562 

0.417 

iniH 

APS- 109  Radar  Homing  and 

5 

0.810 

1.02 

<5 

— 

1 

Warning 

APN  59  Search  Radar 

40 

1.100 

1.680 

35 

1.034 

1.598 

APN-147  Doppler  Radar 

6 

0.956 

1.484 

<5 

-- 

— 

ASN-35  Doppler  Computer 

9 

0.283 

0  528 

6 

0.197 

0.378 

ARN-21  TACAN 

33 

0.739 

0.986 

27 

0.567 

0.732 

APN- 157  LORAN 

18 

0.165 

0.237 

14 

0.208 

0.228 

ASN-24  Navigation  Computer 

14 

0.885 

1.147 

10 

0 . 80J 

1.053 

ADF-73  Automatic  Direction 
Finder 

9 

0.699 

0.574 

6 

0.594 

0.594 

ARC-90  UHF  Communications 

5 

0.996 

1.197 

<5 

— 

— 

Ground  Equipment 

OYK-4  Data  Processor 

12 

1.378 

2.091 

10 

1.073 

1.54 

GSA-51  Display  Conooles 

24 

1.188 

1.638 

24 

1.188 

1.64 

OSH- 12  Recorder/Peproducer 

11 

1.259 

1.555 

11 

1.259 

3..  55 

OSQ-7?  Punch  Card  System 

7 

0.521 

0.412 

7 

0.521 

0.412 

PPS-27  Transmitter 

43 

0.996 

1.426 

42 

0.984 

1.376 

PPS-27  Receiver 

24 

0.656 

i.ie4 

23 

0.645 

1.094 

FPS-27  BCCH 

9 

0.505 

0  602 

8 

0.346 

0.601 

This  procedure1*1  selects  the  most  significant  of  the  independent 
variables,  as  measured  by  the  simple-correlation  coefficients, 
and  then  determines  if  the  addition  to  the  overall  correlation  of 
the  most  likely  remaining  candidate  (determined  by  analyzing 
semipartia!  correlation  coefficients;  is  significant.  If  ther* 
is  a  significant  candidate,  it  is  included,  and  the  process  is 
repeated  until  none  of  the  remaining  variables  can  add  signifi¬ 
cantly  to  the  multiple-correlation  coefficient. 

Because  the  dependent  variable  is  an  average  value  based 
on  samples  of  individual  repair  times,  it  is  reasonable  to  weight 
each  observed  value  of  c2  in  proportion  to  the  sample  size. 
Rigorous  weighting  would  involve  consideration  of  the  variance 
of  a  variance  estimate  to  achieve  required  homos ce das ticity  for 
significance  tests  (assuming  that  necessary  distributional  as¬ 
sumptions  are  satisfied). 

For  practical  purposes,  however,  weighting  was  done  simply 
by  the  square  root  of  the  number  of  sample  observations  for  oaoh 
equipment;,  since  this  tends  to  prevent  disproportionate  weighting 
and  past  experience  using  this  approach  with  reliability-  and 
maintainability-prediction-by- function  analysis  has  proven  favor¬ 
able. 


More  than  20  possible  prediction  parameters  were  evaluated, 
included  were  various  Information-theory  parameters,  design  and 
complexity  parameters,  and  maintenance  design  and  concept  param¬ 
eter?  .  It  was  decided  to  eliminate  from  consideration  any  cal¬ 
culated  regression  equation  that  did  not  have  logical  justifica¬ 
tion.  This  primarily  involved  checking  the  sign  of  the  computed 
regression  coefficient.  For  example,  it  would  seem  logical  that 
a  parameter  highly  positively  correlated  with  conplexity  should 
have  a  positive  coefficient,  indicating  t'~t  as  the  parameter 
value  Increases,  so  does  the  variance  of  repair  time. 

The  presence  of  "wrong"  signs  does  not  invalidate  an  equation, 
because  such  a  result  may  be  due  to  complex  interactions  among  the 
selected  parameters.  However,  past  experience  has  shown  that  pre¬ 
dictions  based  on  such  equations  are  considered  by  some  as  being 
of  doubtful  value.  Since  the  results  obtained  in  this  analysis 
with  "good"  signs  were  favorable,  the  question  became  somewhat  . 
academic . 

Three  equations  were  developed: 

A.  Equations  for  the  variance  of  the  natural  logarithm  of 
active  c orrec t  i  ve -maintenance  time  —  no- trouble- found 
actions  excluded  (see  note  in  Table  XXVHI  for  including 
such  actions. 


xxThe  ARINC  Research  regression  program  uses  the  Square  Root 
Method  described  in  KA  Square  Root  Method  of  Selecting  a  Mini¬ 
mum  Set  of  Variables  In  Multiple  Regression",  Psy  chops  trike, 
Vol.  16,  No.  3,  A.  Suwnerfield  and  A.  Dub  in,  September  I95I. 
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fl)  Prediction  Inputs  —  basic  design  and  maintenance 
parameters.  Sample  size  =21  equipments  (290 
maintenance  actions). 

(2)  Prediction  inputs  —  basic  design  and  maintenance 
parameters  plus  symptom-matrix  parameters .  Sample 
size  =  13  equipments  (176  maintenance  actions). 

B.  Equation  for  the  variance  of  the  natural  logarithms  of 
active  corrective  maintenance  manhours  —  no-trouble- 
found  actions  excluded  (see  note  in  Table  XXVIII  for 
including  such  actions). 

Prediction  inputs  —  basic  design  and  maintenance 
parameters.  Sample  size  =  21  equipments  (290 
maintenance  actions) . 

The  difference  between  A( l)  and  A(2)  is  that  the  latter  re¬ 
quires  development  of  a  symptom  matrix  from  which  the  efficiency 
of  information  transmission,  ET,  can  be  obtained.  Information 
was  available  to  compute  ET  values  for  only  13  of  the  21  systems. 
Equation  A(l)  does  not  Include  symptom-matrix  parameters. 

For  application  to  corrective-maintenance  time,  equation 
A(2)  will  generally  provide  more  precise  results,  but  quantifi¬ 
cation  of  the  ET  parameter  is  somewhat  laborious.  If  a  symptom 
matrix  has  not  been  developed  for  predicting  fault- location  time 
as  prescribed  by  the  procedure  presented  in  Volume  1,  A(l)  may  be 
preferred  in  terms  of  the  effort  Involved. 

The  prediction  parameters  involved  in  the  three  equations 
are  summarized  and  quantified  in  Table  XXVII.  Table  XXVIII  pre¬ 
sents  the  three  prediction  equations  and  necessary  parameter 
constraints  to  avoid  invalid  application.  The  list  of  sample 
equipments  presented  in  Table  XXVI  should  also  be  examined  to 
determine  the  applicability  of  the  sample  equipments  to  the 
equipments  under  consideration.  The  actual  regression  runs  are 
duplicated  in  Appendix  B. 

Various  regression  and  correlation  statistics  pertaining  to 
the  three  equations  are  summarized  in  Table  XXIX.  R*,  the  multiple 
correlation  coefficient  squared,  is  a  measure  of  the  total  varia¬ 
tion  in  the  observed  variance  values  that  can  be  explained  by  the 
variation  in  the  prediction  parameters.  Since  the  calculated  F 
values  exceed  the  corresponding  critical  F  values,  these  values 
are  statistically  significant  at  the  0.05  level.  The  standard 
error  of  estimate,  s,  is  the  standard  deviation  of  the  observed 
(aaiqple)  values  from  the  regression  p'  ^ne.  As  described  below, 
s  is  used  for  confidence- interval  prediction. 


TABLE  XXVII 


PREDICTION-PARAMETER  QUANTIFICATION 


Symbol f  Description 


Efficiency  of  Information 


measure 
of  informa¬ 
tion  presented  by  failure 
symptoms 


Equation  Quantification 


See  Volume  1,  Section  2 


Maintenance  Conclexity 
factor'  -  A  checklist  score 
representing  the  relative 
amount  of  effort  and  time 
required  for  the  preparation 
and  fault-location  activities 


See  note  below 


Relative  Power  Consumption 


e  steady-state  power, 
watts,  consumed  by  the 
equipment  in  its  most  power- 
conruming  mode  of  operation 


1  -  Low 

(0  to  230  watte) 

2  -  Medium 

(250  watts  to  5  kilowatts) 


3  -  High 
(over  5  kilowatts) 


Signal-Data  Handling  -  The 
type  of  circuitry  used  to 
process  a  signal 


Use  sum  of  following  appli¬ 
cable  weights: 

Analog  (Signal)  -  1 
Analog  (Computing)  -  2 
Digital  -  2 


TC  Test  Concept  -  The  equip¬ 
ment-testing  philosophy  for 
on-line  maintenance 


Automatic  Self  Test 
Semi -Automatic  Self  Test  - 
External  Test  Set 
Standard  Test  Provisions  - 

(Use  average  if  sore  than 
one  category  applies) 


NOTE:  To  obtain  the  value  of  M2,  applicable  test  characteristics  from  the 
following  list  are  checked,  and  the  checked  scores  or  weights  (shown 
in  parentheses)  are  totaled. 


Test -Equipment  Requirements 
None  (1) 

One  equipment  (4) 

Two  equipments  (5) 

Three  equipments  (6) 

Maintenance -Manual  Requirements 
None  (1) 

One  manual  (4) 

Two  or  sere  manuals  (5) 

Tool  Requirements 
None  (1) 

Standard  tool  kit  (2) 

Special  tools  (3) 

Both  standard  and  special  tools  (4) 

Special- Handling- Equipment  Requirements 
None  (1) 

Equipment  required  (3) 


Access  for  Preparation 

No  need  for  preparatory  action  (1) 

•  Removal  of  p>:<nel  or  plate  (3) 

Svsptom  Indications 

Obvious  symptoms  present  (1) 

Some  indications  present  (4) 

Kb  clear  failure  Indications  ($) 

Vfarm-Up  Requirements 
Rone  (1) 

Two  to  five  minutes '  warm-up  (3) 
More  than  five  minutes 1  werm-up  (6) 

Cycle-Down  Requirements 
Mane  (1) 

Two  to  rive  minutes'  eye  ‘.e-down  (3) 
More  than  five  .slnute* ' 
cycle-down  (6) 


*rli»  rot-* 


TABLE  XXVIII 


VARIANCE  PREDICTION  EQUATIONS* 


Equation  A(l):  o2inART  =  -0.800  +  0.385(RP)  +  0.221(SD)  +  0.117(TC) 

Constraints:  4  -  RP  +  SD  +  TC  s  10 

Equation  A(2):  a2 /nART  =  0.036  +  0.225(RP)  +  0.023(MC)  -  O.705(ET) 

Constraints:  12  s  mc  s  30 

0.07  s  ET  s  0.70 

Equation  B:  =  -0-7il  +  0.501(RP)  +  0.170(SD)  +  0.133(TC) 

Constraints:  4  *  RP  +  SD  -t-  TC  -  10 


♦Equations  apply  for  predicting  the  variance  when  no-troubla- 
found  actions  are  excluded.  To  predict  when  no-trouble- 
found  (NTP)  actions  are  included,  the  following  average 
relationship  derived  from  the  data  in  Table  XXVI  can  be  used: 

°NTF-IN  “  (1-106)°NTF-0UT 


TABLE  XXIX 

SUIWARY  OP  REGRESSION  STATISTICS 


Equation 

Dependent 

Variable 

Sample 

S".ze 

R 

R2 

Standard  Error 
of  Estimate,  a 

Calculated 

P  Value 

Critical  P 
Value  (5#) 

A(l) 

2 

InART 

21 

0.81 

0.65 

0.256 

IO.65 

3-20 

A(2) 

0^ 

fnART 

13 

0.85 

0.72 

0.210 

7.82 

3.86 

B 

°2inMH 

21 

0.80 

0.64 

0.301 

10.15 

3.20 

To  obtain  a  confidence-interval  prediction,  the  equations  for 
the  confidence  limits  are  as  follows: 

o2l  *  a2  -  tc 
a2v  -  a2  +  k 

where 

Ap  p 

a  is  the  predicted  value  of  c  obtained  from  the  regression 
equation 

2  2 

a  L  and  cj  are  lower  and  upper  confidence  limits,  respecti¬ 
vely,  for  the  variance 


k  =  t 


(a/2,d) 


H 


°n  *i +  2 


f  V-,1  M/2 

L  L  °u  W 

j=I+i 


where 


ra  =  number  of  observations  used  in  the  regression 
analysis 

r  -  number  of  Independent  variables  (prediction  para¬ 
meters  in  the  equation) 

t/  /2  d\  =■  t  statistic  for  a  100  (i-a)$  two-sided  confidence 
1  '  *  ’  interval  based  on  d  ■  (m-r-l)  degrees  of  freedom 
(use  t^a  d>  for  a  one-sided  interval) 

s  *?  standard  error  of  estimate 


computed  Gauss  multiplier 


th 


deviation  where  is  the  value  for  the  i 

independent  variable  in  the  prediction  equation  and 
Xj,  is  the  mean  of  the  observed  values  for  the  1th 


parameter  in  the  sample  data  used  for  the  regression 
analysis 


Table  XXX  presents  the  Information  necessary  for  computing  k. 

Prediction  of  confidence  limits  might  be  desirable  for  in¬ 
vestigative  purposes.  For  example*  a  two-sided  interval  provides 
a  rang*  for  a 2  that  would  provide  information  on  minimum  and  maxi¬ 
mum  expected  sample  sizes  for  a  particular  demonstration  test  under 
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TABLE  XXX 


FACTORS  FOR  CONFIDENCE- INTERVAL  PREDICTION 


Equation 

Dependent 

m 

1 

,  *C-°5»d)  . 

Independent 

Variable 

*i 

Gauss 

Variable 

B 

(90^  Limits) 

Symbol 

i 

Multipliers 

A(l) 

CT2 

^nART 

21 

1 

1.74 

0.2559 

R? 

1 

2.071 

cn  =  0.1011 

SD 

2 

2.130 

e12  =  0.00927 

TC 

3 

2.457 

c13  =  0.01501 
c22  =  0.05615 
c2,  =  0.01845 
c33  =  0.1021 

A(2) 

„2 

0  inART 

13 

I 

1.83 

0.2097 

RP 

X 

1.951 

C11  “  0.3256 

MD 

2 

1.898 

c12  -  -0.01531 

ET 

3 

0.402 

cx,  =  0.2043 
c22  =  0.00256 
c23  =  -0.01105 
c33  -  0.3231 

B 

_2 

/nMH 

21 

3 

1.74 

0.3015 

RP 

1 

2.071 

eu  -  0.1011 

SD 

2 

2.130 

o12  =  0.00927 

TC 

3 

2.457 

c13  =  0.01901 
c22  =  0.05615^ 

c23  -  0.01845 
c33  *  0.1021 

p 

consideration*  An  upper  confidence  limit  may  be  computed  on  a 
and  used  in  the  demonstration  as  a  conservative  practice.  In 
Section  7.5*  specific  uses  of  confidence  limits  on  <t2  are  dis¬ 
cussed  in  connection  wi'tfi  developing  a  prior  distribution  of  0, 
the  expected  value  of  inX. 

7.3.6  Fixed-Sample-Size  Tests ,  Lognormal  Distribution 

7.3.6. 1  Introduction 

The  tests  presented  in  this  section  are  those  in  «hjch  the 
sample  size  Is  specified  and  the  decision  criterion  Is  based  on  a 
lognormal  assumption  for  the  distribution  of  X,  the  maintenance¬ 
time  random  variable  of  Interest.  In  the  Interest  of  generality, 
X  will  net  normally  be  further  identified,  but  it  should  be  clear 
that  X  can  represent  either  corrective-  or  preventive-maintenance 
time  or  man-hours,  and  that  parameters  such  as  mean,  median,  or 
percentile  refer  to  the  specific  time  identification  represented 
by  X. 


7. 3* 6. 2  Test  Number  1:  Test  on  Mean  of  Lognormal 
distribution 


General  Description  of  Test.  A  fixed-sample  test  on  the  mean 
of  a  lognormal  distribution  that  is  based  on  the  asymptotic  nor¬ 
mality  of  the  sample  arithmetic  mean  (central- limit  theorem) . 
Sample  sizes  of  20  or  more  should  be  choBen  to  approximate  the 
asymptotic  distribution  adequately. 

Underlying  Assumptions .  Maintenance  time  can  be  adequately 
described  by  a  lognormal  distribution.  The  sample  size  Is  large 
enough  (say,  at  least  20)  so  that  the  central- limit  theorem 
provides  a  good  normal  approximation  to  the  distribution  of' the 
sample  mean. 

Hypotheses .  HQ:  Mean  ■  u0 

Mean  »  (ux  >  H0) 


Sample  Size.  For  a  test  with  producer's  risk  a  and 
consumer''  a  risk  0, 


n 


*  ZB^l) 

,  2 

'H  -  p-0) 


1) 


p 

where  3  is  a  prior  estimate  of  the  variance  of  the  logarithms  of 
maintenance  times. 


Z0  and  Zg  are  standardized  normal  deviates. 
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Decision  Procedure. 


Obtain  a  random  sample  of  n  maintenance 
compute  the  sample  arithmetic  mean, 


n 


and  the  sample  variance 


Accept  Hg  if  X  i  Pq  +  ZQd//n 
Reject  Hq  otherwise. 

Discussion.  This  test  corresponds  to  the  mean  test  of  Method 
2,  MIL-SSMTl,  except  that  it  provides  a  control  on  both  a  and 
through  proper  choice  of  the  sample-size  value. 

Since  the  mean  and  variance  of  X  are  assumed  to  be  finite, 
by  the  central- limit  theorem,  the  sample  airthmetic  mean,  7,  is 
approximately  normal  for  large  n  with  mean  -  E(x) ,  and  variance  =* 
V\X) .  Hence,  on  the  basis  thal 

v(x)  -  d2  =  e26*a2(eF2-i)  -  n2(e°2-D, 

the  following  are  true: 

5>  2 

under  HQ:  N(uq,  Hq  (e0  -l)/n) 

2 

under  H^j  T  ~  N(p1,  M’12(6C  -l;/n) 

The  distribution  of  7  under  the  two  hypotheses  can  be  represented 
by  the  following  diagram: 

f(X)  , 


Area  ®  0  Area  ■»  a 


It  Is  necessary  to  choose  a  critical  value  C  (HQ  rejected  If 
X  >  C)  such  that  if  (x  =  }lq,  there  is  a  probability  of  a  that 
if  >  C,  and  if  p  *  \l-  ,  the  probability  that  X  s  C  ■  £ 

Thus,  under  the  asymptotic  normality  of  if,  n  can  be  determined 
from  the  following  two  equations: 


^0  +  Zcx  VV(ifn)  =  C 

‘  zp  Ar(ifn)  “  C 


or 


2  /  2 

v-o  +  Vo(e°  -d1/2/^»  -  Hi  -  ZpHite0  -d1/2/vh 


which  yields 


n  =  -LslS — &iL  (e°  -i) 
(m-3.  "  *V 


To  provide  maximum  assurance  that  the  a  and  {3  risks  are  being 
preserved  *;hen  a  prior  estimate  3^  i3  used  in  the  above  equation, 
the  upper  limit  of  any  interval  prediction  or  estimate  on  a2  should 
be  used. 

To  indicate  the  rapid  approach  to  normality  of  the  sample 
mean  if,  100  samples  each  of  sizes  10,  20,  30,  and  I'O  were  generated 
from  a  lognormal  distribution  with  parameters  9  =  3*75  and  a  =*  0.85* 
These  values  are  approximately  the  average  values  observed  In  the 
field  data  obtained  for  21  different  Air  Force  equipments.  For 
each  sample  of  size  n,  if  was  computed  and  the  distribution  of  IOC 
such  values  was  tested  by  the  Kolmogorov-Snimov  Test.  Table  XXXI 
compares  the  theoretical  and  observed  values  of  the  mean  and 
standard  deviation  of  the  X  distributions  and  the  Kolmogorov- 
Smirncv  statistic,  D,  of  thi  maximum  absolute  deviation  between 
the  observed  distribution  function  and  the  theoretical  normal  dis¬ 
tribution  function. 

For  100  samples  of  if  ,  the  hypothesis  of  normality  is  rejected 

at  the  10-percent  significance  level  if  D  >0.122;  at  the  5-percent 
significance  level,  the  critical  region  is  D  >  °  13 6.  From  Table 
XXXI  none  of  the  samples  can  be  rejected  at  the  0.05  level,  but 
the  n  »  20  case  would  have  been  rejected  for  a  «  0.10.  Surprising¬ 
ly,  the  n  ■  10  case  yielded  the  smallest  value  of  D,  but  this  must 
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be  interpreted  as  being  due  to  sampling  irregularities  since  it 
is  known  that  the  approach  to  normality  of  any  3T  distribution  is 
monotonic  in  the  sence  that  as  n  increases ,  the  observed  distri¬ 
bution  more  closely  approaches  the  theoretical  normal. 


TABLE  XXXI 


T,  .ORETICAL  AND  OBSERVED  VALUES  OF  MEANS  AND  VARIANCES 
AND  KOLMCGOROV-SMIRNOV  STATISTICS  FOR  NORMALITY  TEST 


Sample 
Size,  n 

Theoretical 

Mean 

Observed 

Mean 

Theoretical 

aI 

Observed 

°X 

Observed 
Maximum  Devia¬ 
tion,  D 

10 

1 

62.0 

19.9 

19. c 

0.072 

20 

• 

60.3 

14.0 

12.9 

0.131 

30 

61.0 

61.8 

11.5 

11.1 

0.098 

4o 

61.0 

60.2 

9.93 

10.0 

0.119 

Example .  It  Is  dejirod  to  test  the  hypothesis  that  the  mean 
corrective-maintenance  time  is  equal  to  30  minutes  against  the 
alternative  hypothesis  that  =  45  minutes  under  the  lognormal 

assumption.  Previous  data  Indicate  that  cr2  (the  variance  of  in  X) 
«  C.6.  The  a  and  3  risks  are  set  at  5  percent. 

Then:  =  30 

-  45 

If  the  sample-size  equation  is  used,  the  following  is 
obtained : 

n  .  [1.645(30)  +  1.645(45)Jg  (e°,S-l)  s6 
(45-30)2 

7. 3. 6. 3  Test  Number  2:  Test  01  Median  of  Lognormal 
Sisfcrlbutlon 

General  Description  of  Test.  An  exact  fixed-sample  test  on 
the  median  of4  a  lognormal  distribution  that  Is  based  on  the  t  dis¬ 
tribution.  Since  the  test  is  exact,  sample  size  is  restricted 
only  by  the  a  and  p  risks.  Since  the  median  of  a  lognormal  is 

eqjial  to  G3,  tests  on  5T  are  equivalent  to  tests  on  9. 
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Underlying  Assumptions .  Maintenance  times  can  be  Adequately 
described  by  a  lognormal  distribution. 

Hypotheses .  Hq:  Median  =  Kq  or  8  =  6^  -  in  Mq 
H^:  Median  =  M^  or  8  =  0  =  in  M^ 


Sample  Size .  The 
given  "a  and  p  risks  Is 
Terences  at  end  of  the 


sample-size  equation 
derived  in  Reference 
Section) . 


{.resented  below  for 
1  (see  list  of  re- 


n 


(Round  up  to  next 
integer. ) 


where 


a 


t8i  -  ej£ 

^2(yzfv 


fw  P  P 

o  —  prior  estimate  of  a 


Decision  Procedures.  Compute 


f\  1 

8  =  — 
n 

f- 

L  ia  xi 

and 

n 

-i  f  r-1  2  '•? 

s  „  \  (|nXi)  -n8 

1=1 

11  Li-1 

Accept  HQ  if  8  s  8Q 

+  *15-1'  a 

s  //n; 

reject  Hq  otherwise,  where 

„  is  the  100  (l-a)*'il  percentile  of  the  t  distribution  with 
n-  i.  w 

(n~l)  degrees  of  freedom. 


Discussion.  This  test  corresponds  to  Test  3  of  MIL-STD-471 
except  that  control  on  both  n  and  0  is  provide^  through  proper 
choice  of  n.  Since  ft  =  r8,  any  hypothesis  on  M  yields  an  equiva¬ 
lent  hypothesis  on  8  by  the  relationship  8  =  In  M.  Since  9  « 

E(in  X)  and  in  X  is  normally  distributed  with  mean  8  and  variance 
a 2,  the  quantity 


9  -  8 

s//H 

has  the  t  distribution  with  (n-1)  degrees  of  freedom. 

Tims  lognormal ity  and  a  median  specification  yield  the  well 
known  t  test  based  on  a  normal  distribution.  The  sample-size 
equations  are  based  on  the  approximate  normality  of  s.  Details 
of  the  nature  of  the  approximation  are  given  in  Reference  1. 
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i 


ATa.S'.f  r' 

. 


7. 3. 6. 4  Test  Humber  3s  Test  on  Critical  Percentile 

General  Description  of  Teat.  A  fixed-samDle-size  test  on 
the  ^fiff^iLnce^  time  VorrQs'ponclIrig  to  a  particular  percentile 
when  the  underlying  disirlbucion  is  lognormal.  The  decision  cri¬ 
terion  is  based  on  the  asymptotic  normality  of  the  maximum- 
likelihood  estimate  of  a  percentile  value. 

Underlying  Assumptions .  Maintenance  times  can  be  adequately 
de8crT&e3TTy  a  lognormal  distribution.  Under  this  assumption,  it 
is  further  assumed  that  sample  size  is  large  enough  (say  greater 
than  10)  so  that  the  statistic  (e  +  Lsli  is  approximately  normally 
distributed.  p 

Hypotheses .  K0:  (l-p)oh  percentile,  X  =  T..,  or 

P  [X  >  Tq]  =  p 

H^:  (l-p)th  percentile,  Xp  =  or 
P  [X  >  T,  j  =  p,  (Tx  >  T0) 

Sample  Size.  To  meet  specified  a  and  p  risks,  the  sample 
size  to  oe  used is  given  by  the  f ormula 

/2  t  Z  2>.  /Z^j  +  Zq\2 

n  =  ^ - g  P  J  <f“  (^tjt — z~Y'J  ( Round  up  to  next  integer.) 


where 


a  is  a  prior  estimate  of  a4" 

2  is  the  nomal  deviate  corresponding  to  the  (l  -  p)to 
p  percentile 

Decision  Procedure.  Compute 


n 

s  -  i  I  tn  h 

i»l 

r  -«  Z  2  -j-,  / 

X*  "  T0  +  V  [n  +  ST^TTJ 


—  — — . 


where 


Accept  Hq 
Reject  Hq 

Discussion ,  This  test  corresponds  to  the  test  of  *Wet°f 

Test  MIL-STD-471.  As  pointed  out  in  Subsection  7.2-3,  the 
MIL-Sa‘D-471  test  has  an  unspecified  risk  and  its  use  is  thus 
quite  limited. 

The  basis  for  this  test  Is  as  fellows s  Uhder  the  lognormal 
assumption,  the  (l-p)th  percentile  value  is  given  by  Xp  -  6e+<tpa’ 

Taking  logarithms  gives  Xp  *  9  +  ZpOj  therefore,  if  maximum- 

likelihood  estimates  are  used  for  the  'normal  parameters  ®  and  a, 
the  maximum-likelihood  estimate  of  the  (1  -  p)th  percentile  is 

Jin  »  6  +  Z  s 
P  P 

a" 

-  H  A  tn  h 

S 

"  nTT  [£  Xi)2  '  r*®2] 

i*»l 

A 

As  shown  in  reference  2,  p.  58,  Xp  is  approximately  normal  unless 

n  is  quite  snull  (say,  under  5),  with  mean  and  vaviance  given  as 
follows : 

■  E(Xr)  .  E[8  +  Zp»]  -  e  +  zpe  -  xp 


where 


A 

9 


if  9  +  ZpS  £  X*.. 
otherwise. 
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To  meet  the  producer’s  and  consumer’s  risk  requirements,  a 
critical  value  X*  has  to  be  chosen  for  the  sample  estimate  of  the 
(l-p)™1  percentile,  X  ,  such  that 

r  " 

P  [X  >  X»|X  =  Tq]  -  a  araS 


P  [X  >  X*|X  -  -  1-p 


Because  of  the  approximate  normality  of  X  ,  X*  must  satisfy  the 
equation  p 


X*  -  Tc 

•/var  X„ 


=  or 


X*  =  TA  +  Z.,  -/var  X. 


L0  a 


and,  similarly, 


X*  -  \  / - -~ 

*7*===£  »  z  or  X*  »  T,  -  Z  /Var  Xn 
Tvar.  1?  1~P  1  P  P 


Equating  the  right-hand  sides  of  the  above  two  equations  and  sub¬ 
stituting  for  Var  X  (using  n  rather  than  n-1  in  the  second  term) 

yields  the  sample-size  equation 


o2  (StlSf 


1  *>  fX  ' 

JL  0 


Example.  Assume  that  the  following  two  hypotheses  are  to  be 
tested  with  a  «*  |3  =  0.10: 

H0*  X0.05  *  1,5 

Ml!  =  2,0 


Assume  that  o  is  estimated  to  be  1. 
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Then 


and  the  critical  value  in 


»  1.5  +  0.250a 


7. 3*6. 5  Test  Number  4:  Teat  on  Critical  Maintenance  Time 

General  Description  of  Test.  A  fixed-semple-aiae  test  on  the 
probability  that  a  maintenance  action  will  be  completed  before  a 
specified  time  interval.  This  type  of  test  is  applicable  when 
direct  control  on  availability  or  turnaround  time  Is  Important. 

For  example,  if  operational  requirements  dictate  that  an  aircraft 
be  available  within  30  minutes  aftsr  an  initial  alert,  the  prob¬ 
ability  of  completing  any  necessary  equipment  repairs  within  that 
time  period  should  be  high. 


Underlying  Assumptions .  Maintenance  tlpes  can  be  adequately 
described  by  a  lognormal  distribution.  With  the  lognormality 
assumption,  it  is  further  assumed  that  the  statistic  ($  +  Z  a)  is 
approximately  normally  distributed.  p 


Hypotheses .  HQ: 


P[X  >  T]  ■  pn  or  T  «  X 
u  p0 

P[X  >  TJ  -  Pl  or  T  «  Xp  ,  (Pl  >  p0) 


where  T  is  a  specified  time  and 


X  is  the  (l-p, percentile,  i  «  0  or  1 

?i  i 

Note  that  this  test  and  the  preceding  test  have  the  same  null 
hypothesis  Hq  but  differ  in  since  test  number  3  keeps  the  per¬ 
centile  value,  Xp,  constant  and  varies  the  time,  T,  while  this 
test  keeps  T  constant  and  varies  X^. 


Sample  Size.  To  meat  specified  a  and  p  risk  requirements, 
the  sample  size is  determined  from  the  equation 


-0 


+  2VZa  +  Zt 


(Round  up  to  next  integer.) 


where 


Z  Z  +  Z  Z 
a  Pi  P  Po 

ZTTTo . . 

a  p 


and  the  Z's  represent  standardized  normal  deviates. 
Decision  Procedure.  Compute 


9  -  “ 


~  /  4nX, 

n  /,  i 

1=1 


s2  -  5  [  t  (<nXi)2  -  n8a] 


,  Z»ZPi  +  ZPZP0 

— 

Accept  Hq  if  e  +  kB  *  T. 

Reject  Hq  otherwise.  \ 

\ 

Discussion.  This  test  is  an  application  of  the  well  known 
plans  for  acceptance  sampling  by  variables  where  the  quality  crl-  v 
terion  is  baaed  on  percent  defective.  A  discussion  of  this  test 
and  derivation  of  the  equations  for  k  and  n  are  given  in  Reference 
3p  PP.  303-311. 

Example.  It  is  desired  that  95  percent  of  all  repairs  be 
completed  within  1.5  houro.  An  equipment  for  which  only  85  per¬ 
cent  of  repairs  are  completed  in  less  than  1.5  hours  is  considered 
unacceptable.  A  test  is  to  be  conducted  with  a  »  p  =  0.10. 

I 

The  hypotheses  are: 

V  X0.05 
H1J  *0.15  * 
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1.5 

1-5 


Then 


Z0.10Z0.15  +  Z0.10Z0.0f 


J0.10 


J0 . 10 


'l.28Hl.a4)  +  t 1.281(1. 645 


1.34 


Thus  34  observations  are  sampled 
strated  If 


A 

9  +  ka  -  9  + 


m&f 


and  the  requirement  la  demon- 


1.34s  a  1.5 


7. 3* 6-6  Test  N natoer  5:  Teet  on  Joint  Specification*  of 
Lognormal  Parameters11 

General  Description  of  Test.  A  fixed-saiqsle-alze  test  of 
any  one  of  various  possible  pair's  of  lognormal  parameters  such 
as  the  mean  and  percentile.  Table  XDtXII  presents  the  equations 
for  converting  the  values  of  the  two  parameters  specified  to  on 
equivalent  specification  on  8  and  cr2,  the  mean  and  variance  of 
j InX,  a  normally  distributed  random  variable.  Since  the  form  of 
the  critical  region  depends  on  the  relationship  between  the  null 
and  alternative  parameter  values,  the  reasonable  restriction 
that  Oj2  >  <Jq2  was  made  to  siiqplify  the  presentation. 

Assumptions .  The  distribution  of  maintenance  times  can  be 
described  adequately  by  a  lognormal  distribution. 

Hypotheses.  The  null  and  alternative  hypotheses  may  specify 
values  ror  any of  the  following  pairs  of  parameters: 

Mean  -  Variance;  Mean  -  Percentile;  Median  -  Percentile; 

P,th  Percentile  -  P^*1  Percentile;  Mean  -  Median 


Table  XXXII  presents  the  equations  for  converting  hypothe¬ 
sized  values  of  any  of  the  above  pairs  to  an  equivalent  hypothe¬ 
sis  on  6  and  a2.  The  general  set  of  hypotheses  is  then  as 
follows: 

H0:  6  "  e0>  Q2  “  *0 

%'•  6  =  ex,  a2  -  0-j2*  (c^  >«J02) 
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TABLE  XXXII 


PBIATIONSHTp  BETWEEN  9  AND  c2  TO  PAIRS  0? 
SPEC  D  LOGNORMAL  PARAMETERS 


Specified 

Lognormal  Parameters 

Equivalent  Specification 
on  Normal  Parameters 

M  oan  »  |x 

Variance  *  d2 

9  =  top.2  -jin(d2  +  fi2) 

a2  „  l,\£+jr  j 

L  \ir  j 

Mean  «  M- 

pth  Percentile  -  X 
( see  Note)  p 

6  = 

o2  , 

£np,  -  o2/2 

i 

v(zp2  +  S'nfp/Xp))72]2 

Median  * 

pth  Percentile  -  X„ 

P 

6  =* 

o  — 

«nM 

in  (*p/S!)l2 

- „ - - 

_  "P  J 

(l-p.)th  Percentile  -  X„ 

1  P1 

(l~p0)th  Percentile  *  X_ 

^  p2 

Z„  inX„  -  Z„  in  X. 

B  P2  PX  Pj.  Pa 

\  ’  ^1 

\  '  Zpl  / 

Mean  «  ix 

Median  ** 

6  *=  inSf 

<J2  »  2ir.(p./Sf) 

NOTE:  The  Kean-Fercentlle  specification  does  noi,  lead  to  a  p 
unique  lognormal  distribution.  Equations  for  Q  and  cr 
represent  the  more  reasonable  of  two  possible  parameter 
sets.  For  application,  the  following  inequality  just 
be  satisfied:  Z 2  >  2ln  (X^/U). 


test  it  SM*beideterninec'0fron  t«blf°nxn  “«n-percentile 

°12  >  <fq2  means  th»t  requirement 

'£  suar 

sss*s;.  s  g- “JjvSWS&s, r.ss’s's. 

*5  s"V“-25« ,’4^s^1??ssJar^£j?ii? 
£,££ aesjrtbedtn  *5*' for 

The  equations  for  obtaining  n  are  as  follows: 

K  -  (&l  ‘  do)2/°o2>  V  -  ‘xV 

^  *=  */(V  -  l)2,  *  VK/(V  -  l)2 

A0  *  '1  +  *  H,),  Ax  -  V(1  +  2^/(1  + 

B0  =  2(2  +  %)>  Bx  =  2V(1  +  MjJ 

F  =  B0  +  Bx  ,  0  *  AQ  +  A3  +  (A01/2^  -  A11/2Z1^)2 

a  »  (B0  -  Bx)2,  b  =  4(A1Bq  +  AqB1)  -  2FG,  c  =  02  -  4^ 

Then  the  required  sample  size  is  given  by  the  equation 

n  *  ---  tv^b2"  ..  4ac 

varioiis  values**!  ^and^8  for6  thecas  as  °f  these  equations  for 
0.20  v  I0r  Cfte  casss  a  »  g  =  0.05,  0.10,  end 

"he  hTOOt"BUe4t?alMet'S  o  “I  V  *  Salc£l£to‘J  *»» 

SSfftom  t^Uble  b?’iSe?Do?itiChe  V£Ue  °f  n  nan  bs  <»*»«- 

ronliated  K  2dTv®aSL^SP«xt  ^h.^v*’6  for 

values  In  the  table  my  be  JLH!  Sfh8r  l  *^d  next  lower  K 
greater  than  epecmS!  cnsure  tnat  the  ri8ks  «•  «> 
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tabl i  min 


SAMPLE  SIZES  TOR  VIET  HOMS©  5 

FOR:  a  -  p  -  0.0$,  0.10,  0.20  AND  FOR  \AKICU3  A  AMD  V  VALUES 


a  *  0.05,  p  -  0.05 


DQ 

m 

1.3 

EB 

eb 

D 

EQ 

B3 

EB 

m 

fSU 

EQ 

1.2 

CT 

1.5 

1.6 

EB 

D 

Bl 

S9 

0.03 

45& 

311 

209 

lb9 

in 

87 

71 

59 

51 

44 

0.12 

91 

86 

78 

69 

6l 

54 

48 

43 

39 

35 

3.04 

326 

2b6 

175 

133 

103 

82 

68 

57 

49 

43 

0.14 

79 

75 

70 

63 

56 

50 

45 

41 

37 

34 

o.ob 

253 

20b 

157 

121 

96 

78 

65 

55 

48 

42 

6a 

67 

63 

57 

52 

47 

42 

39 

35 

32 

0.05 

207 

17* 

135 

in 

89 

7b 

62 

53 

46 

4l 

0.l8 

62 

60 

57 

53 

48 

44 

40 

37 

34 

31 

0,06 

175 

152 

125 

102 

8b 

70 

6o 

52 

45 

4o 

0.20 

56 

55 

52 

49 

45 

41 

38 

35 

32 

30 

0.07 

152 

135 

lib 

95 

79 

67 

57 

50 

44 

33 

0.25 

45 

45 

43 

41 

39 

36 

34 

32 

30 

28 

0.0& 

13b 

121 

104 

88 

75 

6b 

55 

48 

43 

38 

0.30 

38 

38 

37 

36 

34 

32 

30 

29 

27 

25 

0.09 

120 

no 

96 

83 

71 

6l 

53 

m 

42 

37 

0.35 

32 

32 

32 

31 

30 

29 

28 

26 

25 

£2 

0.10 

109 

97 

89 

76 

67 

59 

51 

9 

u 

I37 

0.40 

28 

28 

?8 

23 

* 

27 

26 

25 

25 

a 

SI 

■ 

a 

-  o.io,  p  = 

.10 

m 

1.2 

SI 

m 

Si 

EQ 

1.7 

1.8 

i 

51 

m 

m 

m 

Bl 

tH 

B9 

m 

S3 

m 

3.01 

467 

256 

153 

102 

7b 

57 

46 

38 

32 

28 

0  12 

56 

53 

48 

33 

30 

26 

si 

a 

0.03 

278 

189 

1128 

91 

68 

53 

bb 

36 

31 

27 

0.14 

48 

46 

42 

31 

28 

25 

a 

a 

0.03 

198 

150 

100 

81 

«3 

50 

42 

33 

30 

m 

0.16 

42 

41 

38 

29 

26 

a 

a 

si 

0.0b 

15b 

12b 

96 

7* 

55 

A8 

40 

3b 

29 

m 

o.i8 

38 

37 

35 

32 

30 

27 

25 

a 

a 

SI 

0.05 

126 

106 

85 

68 

55 

b5 

38 

33 

29 

25 

0.20 

3b 

33 

d2 

30 

28 

25 

a 

a 

a 

25 

c.cq 

*57 

V* 

66 

Ho 

*0 

27 

2£ 

0.22 

31 

31 

30 

28 

26 

a 

a 

a 

.2 

SI 

u.10 

66 

6l 

-  _J 

55 

36 

28 

25 

m 

0.24 

29 

28 

27 

_ 

25 

a 

2£ 

a 

a 

a 

£ 

a 

=  0.20,  p  • 

0.20 

Vs 

m 

sg 

1.15 

BE 

IBS 

m 

1.35 

1.4C 

1.45 

IB 

IBS 

m 

Hr 

IBB 

1.30 

1.35 

1.4C 

1  45 

1.50 

O.OC5 

*66 

30b 

197 

i> 

97 

7b 

58 

48 

40 

34 

0.040 

71 

67 

H 

47 

42 

37 

32 

29 

26 

0.010 

258 

201 

lb9 

111 

85 

167 

5* 

45 

38 

33 

0.045 

63 

60 

IS 

44 

39 

35 

31 

28 

25 

0,015 

179 

150 

120 

9* 

75 

61 

50 

42 

36 

31 

0.050 

57 

55 

46 

4l 

37 

33 

30 

27 

£2 

0.030 

120 

100 

82 

67 

55 

47 

ko 

34 

30 

0.060 

48 

46 

44 

40 

37 

33 

30 

27 

25 

22 

0.035 

loo 

86 

72 

6. 

51 

bb 

37 

33 

29 

0.070 

4i 

40 

38 

36 

33 

30 

28 

25 

a 

22 

0.030 

86 

76 

^5 

56 

43 

41 

36 

31 

28 

0.080 

36 

35 

:,4 

30 

50 

28 

v6 

22 

a 

£2 

0.035 

80 

75 

67 

59 

51 

bb 

39 

34 

30 

27 

0.090 

32 

32 

31 

79 

27 

26 

'sl 

£2 

a 

a 

0.100 

29 

29 

28 

27 

25 

a 

a 

n 

a 

£2 

Hot«  —  A  line  under  2$  25)  al#nlfia»  that  ■  lower  sample  alr»  than  25  nay  be  used  but 

35  li  re  cow  ivied  ae  a  unlnum  for  M  demonstration . 


Decision  Procedure.  The  decision  procedure  io  as  follows: 
(1)  Obtain  a  sanpls  of  n  observations  of  maintenance  times. 


Xp  Xg,  xR. 


(2)  Corpute  Z  =  -ij  )  (InXj  -  p)2  *  -Ay  fs2  t  (Y  -  p)2] 

. —L  n_  L  J 


where  p  = 


u0  i=l  °0 

Vi2  -  W  v  -  ei 


°r  -  v 


v  -  1 


y  =  h  I  inXi 

i=l 

b2-  i[  £  (**t)2  -  n?2J 


(3)  Compute 


^  - 


(9o  -  P)‘ 


(V  -  1)' 


1  +  2M.  (1  +  VL)‘ 

u  ■  mg*»  v  -  n  r-ra^ 


(4)  Conpute 

C  =  |(za  +  JzT~)2 

(5)  If  Z  ^  c,  Hq  is  rejected 

If  Z  <  C,  Hq  is  accepted 
12 

Discussion.  The  basis  of  this  test  is  developed  from  the 
Neymtm-Pearson  lemma,  which  states  that  the  test  critical  region 
for  rejecting  Hq  consists,  of  points  in  the  sample  space  such  that 

"(X|Ho> 

mnqr*** 

12.  For  further  details  see  M.  Kendall  and  H.  Stuart,  Advanced 
Theory  of  Statistics.  Vol .  2,  Griffin,  1961,  pp.  17^-17^  227*229 


»fr«THESiCfeB*  WBKE  J! 


where 


X  -  (Xp  Xg,  ...,  Xn),  the  sample  observations 

L(X|Hj)  is  the  likelihood  of  the  n  sample  observations  if 
Kj(j=0,l)  is  true 

ka  is  a  constant  ( >0)  such  that  the  a -risk  is  satisfied 
If  Y±  =  JJnXp  then 

_n 

L(l!Hj)  =  (V5F  0j)‘n  (Yj  -  Sj)2],  j=0,l 

2  2 

and  for  cr^  >  .  the  above  likelihood-ratio  inequality  can  be 


shown  to  reduce  to 


n 


1  (yl  -  P>2  *  cc 

i=l 

whef;e  cQ  is  a  constant  independent  of  the  observations. 


Since  Y~  N^0o,Oq  )  under  Hq-  (Y  -  p)/aQ 

2 


N 


(eo"fJ^//°o,:L 


n 


Y  -  p- 

Then  it  can  be  shown  that  )  { — - - )  has  a  noncentral  chi- 

<->  \  ao  J 


i=l 


square  distribution  with  n  degrees  of  freedom  and  noncentrality 
0  “ 

parameter  n  ^ — - — - J  =  nM^.  Thus  a  test  of  size  a  is  obtained 

by  finding  the  100  k(l  -  a)  percentile  of  a  noncentral  chi-square 
distribution  with  n  degrees  of  freedom  and  noncentrality  param- 

/  O 

eter  nJ^.  If  this  percentile  point  Is  denoted  by  X  (nM^ci), 
the  decision  criterion  is  as  follows: 

n  y  -  p 

Reject  Hg  if  y  (~~j  ‘  Z  - 


i=l 


where 


p[z  s  X'2(n»\),o)|^) 


=  a 


152 


and 


,2 


Z  ~  Xn  (nMo)i  a  rioncentral  chi-square  variate. 


The  power  of  the  test  Is  then  defined  as 


1  -  p  =  P 


Z  *  *n2(nM0’a)lHl 


and  thus  the  distribution  of  Z  under  is  required.  Since 
(Y  -  p)  ~  -  p,  a^2)  ander  then 


^  -  P\ 

L 

i=i 


(*\  ~  P\  a0  2  /®i  "  P\ 

\rpr~)  =  — ^  z  -  xn  (nMq)*  where  ^  aj~-) 


Hence,  the  power  function  is 

2 


■P 


=  p[^  0  z  4  _0^  x^2(nM0,o)]H1l 

al  °i  J 

=  pfx^fni^)  *  ^x'2(nM0,a'i|H1j 


/  2 

If  Xn“(^)  is  a  noncentral  chi-aquare  variate  with  n  degrees 
of  freedom  and  noncentrality  parameter  A,  then  let 

,  n+  Z\  (n  t  M2 

n  +  X  *  n  +  2'a 


Then  for  v  over  30, 


\  U 


-  (2v  -  1)^2  ~  N(0,1)  approximately.  * 


From  this  approximation,  the  following  a’-e  obtained  from 
the  a  and  1  -  p  equations 


x;2("Va>  -  T-  (za  +  ’^T“1>2 

X^{M0,a)  =  V  ^  (q  .  &+ 

1^3 


where 


U1  = 


1  + 

rr-Rf  >  vi 


(i  +  a,) 

vi  “  n-"T  —  -3(1  ,  i  =  0,1 


Equating  the  right-hand  sides  of  the  above  two  equations  leads  to 
tne  equation  for  n. 

Example,  assume  that 

Hq:  Mean  =  0,5,  95tn  Percentile  =  1.5  hrs. 

Mean  =  0.8,  95th  Percentile  =  2.5  hrs. 


a  =  6  =  0.05 


Equivalently, 


Then 


Hg:  G  =  -1.128,  a"  =  0.869 
Kx:  0  =  -G.715,  o2  =  0,?8k 


2 

"  eo)  (-0.715  +  1.128)2  „  , 

“  - n —  -  - — ~-'*k-irrx — —  ~  0., 


v  1  _  0.984  _ 

v  -  7-5  -  055  “  1-13 


From  Table  XXXIII,  for  a  “  fl  «  0.C5,  K  =  0,20,  and  V  =  1.1, 
the  required  sample  size  is  found  to  be  equal  to  5o  observations. 


7.3.7  Sequential  Tests  -  I 
7-3.7. 1  Introduction 


Distribution 


Three  different  sequential  attainability-demonstration 
tests  are  presented  in  this  subsection: 

•  Test  on  a  joint  specification  of  lognormal  parameters 

•  Test  on  the  mean,  median,  or  percentile  of  a  lognormal 
distribution 

D 

•  Test  on  a  critical  mair scnance  time,  a~  unknown 


^  ’.w. 


7-3 ‘7 Test.  Humber  6:  Sequential  Teat  on  Joint  Specifics- 
tlonof  Lognormal  Parameters 

Description  of  Teat .  This  test  Is  the  sequential  analog  of 
Test  rhuaber  5l  in  which  any  one  of  various  possible  pairs  of  log- 
normal  parameters  nay  be  specified.  Table  XXXII  is  used  again  tc 
convert  the  original  specifications  to  one  in  terns  of  9  and  a2., 
the  mean  and  variance  of  InX,  which  is  normally  distributed. 

Assumptions.  Maintenance  times  can  be  adequately  described 
by  a  lognormal  distribution.  Simple  random  sampling  is  performed. 

2  v 

The  original  specification  is  such  that  o^  >  o0". 

hypotheses.  Any  of  the  following  pairs  of  parameters  may  be 
spec if led: 

Mean  -  Variance;  Mean  -  Percentile;  Median  -  Percentile 

P1th  Percentile  -  P^h  Percentile;  Mean  -  Median 

Table  XXXII  presents  the  equations  for  converting  hypothe¬ 
sized  values  o£  any  of  the  above  pairs  to  an  equivalent  hypothe¬ 
sis  on  9  and  a6.  The  general  set  of  hypotheses  Is  then  ao 
follows: 

r 

•V  «  =  80,  o2  .  c02 

Hl:  6  =  61'  *2  *  °1  (ffl2  >  aQ2) 

with  specified  a  and  p  risks. 

Sample  Size.  The  sample  size  for  a  sequential  test  is  a 
random  variable.  For  the  hypothesized  pairs,  the  expected  values 
of  n  are  as  follows: 

E(n| 9  =  00,  c2  -  o2)  =  —-Z-3ULZ31  J&±*35*L- - — 

ir(^z)  +  1°1  ■  ao2  ■  f®i  -  a0)2j 

E(nj6  ■=  9X,  a2  «  o^)  - 2 — gift*1*5  f  . — 

+  ^2  [°l  ■  a02  +  (®1  '  eo'2j 

where  A  *  1  *■  -P  ,  B  ■  y 

a  1  -  a 
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Dec la Ion  Procedure,  The  decision  procedure  Is  as  follows: 
(1)  Compute 


(•i  '  *o> 


C1  ~  °0 


1  m  2  2  2 

.  ffl  al  "  a0 

+  in  —  g  ,  D  =  ■  "■— -g  — g  ,  E  - 

ff0  a0  01 


@iOq  —  6q(X^ 


an  -  Or 


(2)  Compute 

an  =  2D|nB  +  CDin  for  m  =  1,  2,  3,  ... 

ba  *  2D4nA  +  CDm  for  m  =  1,  2,  3,  ... 

Random  samples  of  maintenance- basic  times  X1,  X2,  ...  are 
obtained  as  long  as 


s<£ 


(**i  -  E)^  <  bm 


A  decision  is  made  the  first  time  the  above  inequality  is  violated. 


Thus: 


m 

r1  2 

Accept  Hg  after  m  observations  if  ^  (inX^  -  E)  s  am  ■ 

N  1*1 


Reject  Hq  after  a  observations  if  >  (JnX|  -  E)^  a  bm 

i=l 

Continue  testing  otherwise. 

Since  the  acceptance  and  rejection  boundaries  are  linear 
with  m,  the  number  of  sample  observations,  a  graphical  procedure 
la  easily  established  (e.g.,  in  Figure  2,  Section  IV). 

Discussion.  The  test  hypothesis  and  the  use  of  sequential 
eaaplTng  corresponds  to  Test  Method  1  of  MIL-STD-471.  However, 
the  latter  is  based  on  converting  a  mean  and  percentile  specifi¬ 
cation  to  two  percentile  specifications  and  then  employing  a 
binomial- type  sequential  test.  This  test  makes  direct  use  of  the 
lognormal  assumption  and  therefore  should  have  better  efficiency 
in  terns  of  sample  size  for  the  comparable  MIL-STD-kyi  test. 

The  development  of  the  decision  criterion  ie  a  direct  result 
Of  application  of  Wald’s  theory  cf  the  sequential  probability- 
ratio  test. 


iOv'SA'. 

■&r  s’ 
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,  ^2^2-  Consider  the  example  of  Test  Nuafcer  5  vhtsre  «mer*i. 
fled  mean-percentile  values  ledlo  tbu  hypotheses  P 

H0:  9  "  ”1*128*  ®2  M  0.869  and  1^:  9  -  -O.T15,  a2  -  0.984 
From  the  sample- size  equations,  it  is  found  t!*t 
E(nje  «  d0>  a  -  a02)  -  29,  and  B(n|fl  .  B  ,  o2  -  ff  2)  *  26.  This 
corresponds  to  a  fixed  sarnie  size  of  56 


7. 3. 7. 3  Test 

nmaUH 

or  R 


niter  7:  Sequential  Test  on  the  Mean.  Median* 
entile  o^k  liOgnoraaX  Sis  tVibut lon^  ■ 

convJlffffSf*??  £LSffls;4.i.*?,Uentt%?-  teot  ttmt  u  s*”<!  ™>  » 

lognorusd  parameter  to  an  topothe- 
sis  on  9  .  E(JnX) .  a  mean  or  percentile  specification^* 

prior  estimate  of  c \  is  required  for  the  conversion.  for  a  aedl- 

baaSe2iffe£tiS5i«?  ,?*,ed  1?c  Mtlaatod.  Since  the  test  is 

~*?®d  on  ***  asyaptotic  sufficiency  of  noxiaua-llkelihood  estl« 

Bl"  ,hTOld  ba  •««-/«  wro^Sit. 

■ lytng t Asgugptiona .  Maintenance  tiaes  can  be  adeowtaly 

dee-rlbed  by  a  loghoraai  distribution.  Sinple  random  saacllrjr  is 
performed,  and  the  minimum  cample  size  is  large  enough*»o?tltet 
s  is  a  good  estimate  of  c2. 

hypotheses .  (The  prior  estimate  of  o2  is  denoted  by  a2). 

man  Specification  Hq;  »  «  ^  -e  0  .  ^  -  ^/2 

■/ 

V  U  *  -  o®/2 

Percentile  Specification 

V  *p“  T0  •*  ®  *  ®0  *  lnT0  -  2  a 


p' 

T^  '•*  <t  m  =.-  2nT^  -  ZpO 


Median  Specification 


H. 


I* 


/v 

M 


M 


Mo  0  ,  eQ 

"►  0  X  © 


4nJ^ 

inM^ 


^  Size.  The  sample  size  is  a  random  variable  and  is 

therefore  usually  evaluated  ic  terms  of  expected  valuer  Be« 
cause  of  the  asymptotic  nature  of  this  te£t,  howew,  a'miSLum 


number  of  saplea  should  be  specified.  Values  of  n  greater  than 
25  should  prove  satisfactory  (see  Discussion  section).  The  symbol 
n*  is  used  to  denote  the  specified  minimum  sample  size. 

Decision  Procedure.  Randan  samples  of  maintenance-task 

•  •  •  •  t  1  •  •  >i  are  taken  as  long  as 

the  following  Inequality  holds: 

„  2 


a-**-  Jr3  +  m 
81^0 


where  / 


a 


B 


-J- 

1-a* 


e 


a 


1  m 
-  Z 

*  i-1 


IriX 


i* 


Accept  Hq  if 


m 

Z 

i-1 


fnX 


i  v  S^6q 


Ink  +  m 


90+9l 


bB,  (e  a  n*) 


m 


i  [J,  («i>*  *  ">9/]’ 


and 


m 

Z 

i-1 


IriXf  s  for  some  min* 


Reject  Hq  if 


m 

£ 

i-1 


InX.  i  b,  for  some  min* 
1  m 


Discussion.  Comparison  of  the  decision.critorion  of  this 
test  with  a  sequential  test  on  the  median — 0 r  known  reveals  that 
the  only  difference  is  that  o2  is  replaced  by  s2  in  the  accept/ 
reject  boundaries.  The  basis  for  this  substitution  is  discussed 
In  Reference  4,  pp.  54-56. 

To  Indicate  the  acceptability  of  this  procedure,  the  sequen¬ 
tial  test  was  simulated  on  a  conputer.  The  test  hypotheses  were 
H  -  30  minutes  and  H^:  S!  *  45  minutes  at  o  -  p  *  0.10.  The 

minima  saipXs  size  before  a  decision  could  be  made  was  set  at 
20.  Lognormal  random  mudbei'S  with  medians  of  30,  36.75,  and  45 
minutes,  and  with  a2  -  1.44,  were  generated.  For  250  simulated 
tests  at  each  median  value,  the  theoretical,  and  observed  results 
are  as  follows: 


For  the  three  cases  examined,  the  observed  acceptance  prob¬ 
abilities  are  slightly  under  the  theoretical.  Because  of  the 
minimum-sampie-sise  restriction,  the  average  number  of  observa¬ 
tions  is  somewhat  hig'isr  than  theoretically  expected. 

7.3.7.^  Teat_ Number  8s  Sequential  Test  o?  Critical  Mainte¬ 
nance  timerTbgnoriaal  pTatrlbation,  HTnfcilbwn 

Description  of  Teat.  A  sequential  test  of  the  proportion  of 
a  normal^p opuTaEToii  exceeding  a  given  constant.  The  assumption 
of  a  lognormal  distribution  of  maintenance  times  permits  direct 
application  to  testing  the  percentile  value  of  a  critical  mainte¬ 
nance  time.  This  test  is  an  approximation  of  the  sequential  t 
test  {see  Discussion  section). 


Underlying  Aaawoptfons „  Maintenance  times  can  be  adequately 
described  by  a  lognormal  distribution.  Simple  randora  sampling  is 
performed.  The  decision  criterion  is  based  on  tte  approximate 
normality  of  the  Statistic  X  +  ks  when  samples  are  taken  fro®  a 
normal  distribution. 


Hypothesis. 


Hq!  P  [X>T]  -  P0  or  T  »  3^ 
Hx:  P  [X>T]  -  px  or  T  «  Xp 


/ 


where  T  Is  a  specified  critical  maintenance  time. 

Semple  Size.  Since  n  is  a  random  variable  in  a  sequential 
test,'  no  sample  sice  is  specified. 


Decision  Procedure.  The  decision  procedure  is  as  follows* 

(1)  Compute  A  =  In  ,  B  ■*  in 

(2)  Compute  the  acceptance  boundary 


m 


where 


m 


A  D  ^  2A 

(m-l)(Z  -  Z„  )  ’  m  4Zr 


P0  Pi'  P0  ?1 

(3)  Compute  the  rejection  boundary 


-  -  (Z  +  ) 

Z_  )  Po  pl 


fe, 


y^v1.  - 1 


m 


rm 
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where 


B 


2B 


,  F  - — —  -  (Z„  +  Z_  , 

(b-1)(Z  -  2  )  ra  m(Z  -  2  )  p0  *1 

P0  Pj  P0  PX 

(4)  After  each  observation,  compute 


m 

T-  £  inX. 
i«l  2 


m 


s. 


f  m  —  2, 3,  *  • 


m 


where 


m  m- 


m 


2  (inX.)  - 
1«1  x 


/  m  \  r\ 


m 


K^!V  Ho  is  accepted.  If  k^.  z  bm,  Ho  is  rejected, 
»  %  <km<  bra#  another  maintenance  task  is  sampled. 


Discussion.  This  test  is  the  sequential  analog  of  the  fixed- 
sample  test  for  critical  maintenance  time  and,  accordingly,  should 
require  smaller  sample  sizes  on  the  average,  except  possibly  for 
cases  in  which  Xp  <  T  <  .  The  test  described  here  is  an  ap¬ 

proximation  to  the  WAOR  sequential  t  test  (see  reference  5),  which 
requires  the  use  of  tables  of  the  noncentral  t  distribution.  The 
derivation  of  the  test  Is  given  in  Reference  2,  pp.  83-85. 

7.3.8  Nonparametric  Tests 
7. 3. 8.1  Introduction 


Nonparametrlc  tests  have  the  desirable  characteristic  that 
it  is  not  necessary  to  assume  an  underlying  distribution  with 
regard  to  the  maintenance-time  random  variable  of  interest.  How¬ 
ever,  they  generally  require  greater  sample  sizes  than  correspond¬ 
ing  parametric  or  nondistribution-free  tests . 


Five  different  nonparametric  maintainability -demonstration 
tests  are  presented  in  this  subsection  (the  first  four  are  fixed- 
ssmple-aize  tests) s 


(1)  Test  of  median  or  percentile 

(2)  Test  of  critical  maintenance  time 

(3)  Test  of  two  critical  maintenance  times 
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(4)  Test  of  specific  lognoraal  distribution 

(5)  Sequential  test  of  critical  maintenance  time 

7. 3 -8. 2  Test  Humber  9  -  Test  of  Median  or  Percentile 

Description  of  Test.  A  ncnparsmetrlc  test  of  a  median  or 
percentile.  Two  specific  sets  of  hypotheses  are  considered.  One 
Is  for  the  case  in  which  a  desirable  median  or  percentile  is 
specified  such  that  there  should  be  a  high  probability  of  accept¬ 
ance  if  the  actual  maintenance- time  distribution  conforms.  This 
is  the  usual  null  hypothesis  with  an  associated  a  risk.  The  other 
case  offers  the  consumer  protection  against  accepting  a  system 
that  has  the  median  or  percentile  equal  to  a  specified  value  re¬ 
presenting  an  undesirable  maintenance  level.  The  test  statistic 
for  either  case  is  based  on  the  number  of  observations  exceeding 
the  specified  time. 

Underlying  Assumption.  Since  the  test  is  nonparametrlc,  it 
is  not  neceesary  to  make  an  assumption  concerning  the  distribu¬ 
tion  of  maintenance  times. 

Hypotheses.  Test  I  Test  II 

V  *p-To  «o!  *p<Ti 

Hr  ^>*0  «r  VTi 

For  test  I,  TQ  represents  a  desirable  value  for  the  (l-p)^ 
percentile;  e.g.,  Hq  might  be:  median  «  30  minutes  and  therefore 
H,  is  the  composite  alternative,  median  >  30  minutes.  For  test 
II,  T^  represents  an  ’indesirable  value  for  the  (1-p)  percentile; 
e.g .,  Hj  might  be:  median  *  45  minutes  and  Hq  is  therefore  median 
<  45  minutes.  Note  that  the  hypothesis  H-^s  «  T,  of  tost  II 
is  the  same  form  aa  the  Hq  hypothesis  of  test  I. 

Sample  Size .  Since  both  test  I  and  test  II  contain  a  com¬ 
pos  itoT^po^TieiTs  end  therefore  only  one  specified  risk,  there  is 
no  sample- s lie  restriction  in  the  usual  sense.  If  r  Is  the  number 
of  maintenance  times  exceeding  the  specified  value  and  c  Is  the 
acceptance  number,  sn  accept  decision  Is  made  if  r  s  c  when  n 
maintenance  times  are  observed.  For  a  given  c  value  (c  »  0,1,2, 
n  is  found  from  the  following  equations,  which  employ  the 
binomial  distribution: 

f  (?)  Pr  (1-P)n’r  *  1-a 


Test  X: 


-  t’  *  r  ,-■ 


Test  II; 


I® 


(i-p)' 


Table  XXXIV  presents  the  sampling  plans  (sample  size  n  and 
acceptance  number  c)  for  various  p,  a,  and  £  values  for  c  =  0,1, 


tccepti 

5,3,4, 


and  5. 


\ 


TABLE  XXXIV 

SAMPLE  SIZES  FOjK  ‘TESTS  I  AND  u  FOR  VARIOUS 
PERCENTILES  AKD  RISKS  FOR  c  -  0  THROUGH  5 


Percentile 


.20  8  15  20  39  39 

.10  7  17  16  45  32 

.05  6  18  14  50  27 


*  Risk  requirement  cannot  be  satisfied 


In  general,  the  plan  with  the  higher  c  number  offers  better 
protection  against  accepting  a  poor  product  (for  vest  l)  and 
rejecting  a  good  product  (test  II);  therefore,  c  should  be  made 
as  large  as  possible,  consistent  with  constraints  on  sample  size. 

Decision  Procedure .  A  random  sample  of  n  maintenance  times 
X^,  %£,  .  .  .,'X  is  observed,  and  a  count  Is  taken  of  the  number 

of  such  times  that  exceed  the  specified  time  T.  This  number  is 
called  r. 

For  test  I,  H0  is  accepted  if  r  s  c  and  is  rejected  other-wise. 

For  test  II,  H,  is  accepted  if  r  s  c  and  is  rejected  other¬ 
wise. 


Discussion.  Test  II  Is  equivalent  to  Plan  4  of  MU- STD-471 . 
Test  I  is  an  alternative  in  which  a  desirable  percentile  value 
is  specified.  The  choice  between  I  and  II  dependB  on  whether  it 
is  better  to  control  the  a  or  the  p  risk,  which  In  turn  depends 
on  the  costs  associated  with  each  of  the  possible  wrong  decisions. 
The  next  test  described  has  a  control  on  both  a  and  fi  risks  and 
is  therefore  recommended  as  a  better  alternative  because  it  in¬ 
cludes  the  specified  hypotheses  of  this  test. 

7. 3. <i.3  Test  Number  10  -  Test  of  Critical  Maintenance  Time 

Description  of  Test.  A  nonparametric  test  of  a  critical 
maintenance  time  and  associated  percentile  value.  An  exomple  is 
the  following  set  of  hypotheses:  Hq  —  30  minutes  is  the  median 

(50th  percentile);  and  --  30  minutes  is  the  25th  percentile. 

In  this  test  both  the  null  and  alternative  hypotheses  refer  to  a 
fixed  time  and  the  percentile  value  varies.  In  the  preceding 
test  the  percentile  value  remains  fixed  and  the  time  varies. 

As  In  the  preceding  test,  the  number  of  maintenance- time 
observations  exceeding  the  critical  time  Is  compared  with  an 
acceptance  number  c  to  determine  acceptance  or  rejection  of  the 
null  hypothesis. 

Underlying  Assumptions .  No  specific  assumption  Is  necessary 
concerning  z he  als tr i butTon  of  maintenance  time.  In  the  develop¬ 
ment  of  the  equations  for  determining  the  decision  criterion  and 
sample  size,  the  normal  or  Poisson  approximation  to  the  binomial 


T  = 


T  » 


distribution  is  used. 
Hypothesis. 


Hr 
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‘  v  ■ 


Ui\ 


k  ; 


xxmm/Earzrsp 


Sample  Size,  n,  and  Acceptance  Number,  c.  The  normal  ap¬ 
proximation  to  the  binomial' distribution  is  employed  to  find  n 
and  c  when  p0  is  not  a  small  value.  Othsrwise,  the  Poi3son  ap¬ 
proximation  is  employed.  The  equations  for  n  and  c  are  as 
follows : 


For  0.20  S  pQ  i  0.80 


n  = 


c  =  n 


Z6  feA  +  z°  /Wo 


Pi  -  Pr 


zb  po  /Mi  +  Vl  /Wo 

Za  /pOqO  +  Zg  /plql 


(Use  next  higher 
integer  value „ ) 


(Use  next  lower 
integer  value . ) 


For  pQ  <  0.20 

For  this  case  n  and  c  can  be  found  from  the  following  two 
equations: 


v> 

L 

r=0 

i 


g-npp  (npQ)] 
r! 

0-npj^  (npx)] 
rl 


1  1  -  a 


Table  XXXV  provides  sampling  plans  for  various  a  and  p  risks 
and  ratios  P]/P0  when  pQ  <  0.20. 

Decision  Procedure.  Random  samples  of  maintenance  times  are 
taken,  yielding  n  observations  X^,  X^,  ...,  Xn>  The  number  of 

such  observations  exceeding  the  specified  time  T  is  counted.  This 
number  is  called  r. 

Accept  Hq  if  r  sc. 

Reject  Hq  if  r  >  c . 

Discussion.  This  plan  corresponds  most  closely  to  test 
method'  4  of  M'E-STD-471  except  that  both  an  a  and  p  risk  are 
specified.  It  is  preferred  over  the  preceding  plan  oecause  It 
provides  known  protection  to  both  the  producer  and  consumer. 
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An  example  of  developing  the  decision  criterion  for  the  tall- 
percent  lie  hypothesis  is  provided  in  Table  XXXV.  For  an  example  of 
&  median  specification,  assume  that  the  following  hypotheses 
exist,  with  a  -  (5  -  C  10: 


30  minutes  -  X^  ^ 
H^:  30  minutes  *  XQ  ^ 


median 

25th  percentile 


Then  Z„  *  Za  »  1.28  and 
a  0 


n .  (i.28)2  [djaL^^jiagiLsai8 


c  *  23 


-JS211 


•50)(.50)  ^  /(.75)( .25) 


The  actual  probabilities  corresponding  to  1-a  ■  0.90  at 
P0  ■  0.50,  and  to  0  »  0.10  at  p-^  »  0.75 ,  from  tables  of  the  bi¬ 
nomial  distribution,  are  0.895  and  0.096,  respectively  —  an 
excellent  agreement. 


7.S.8.4 


Description  of  Test.  A  nonparametric  test  in  which  two 
tritic^  maintenence  t'frae s  are  specified  and  separate  tests  are 
applied  to  each.  The  two  critical  maintenance  times  will  generally 
be  a  median  and  value.  An  overall  accept  decision  is  made 

only  If  both  individual  tests  are  passed,  As  in  tests  9  and  10, 
the  number  of  observed  maintenance  times  exceeding  specified 
values  is  the  statistic  used  to  mav-»  the  accept /reject  decision. 

,  No  specific  assumptions  concerning 
penance  times  are  necessary. 


-  Test  of  Two  Critical  Maintenance 


SD&ttgfiSSL' 


5  “  V  * "  \ 


Pq^Pi * ^i >  p^>Q^, 

i  »  0  or  1) 


V  *-*,»  tmh. 


'  ■*  :>L'i 


An  example  of  the  hypotheses  is  as  follows: 

Hq:  30  minutes  »  median  (Xq  ^q), 

60  minutes  -  95th  percent  He  (Xq  05) 

Hj_:  30  minutes  -  35th  percentile  (Xq  g^), 

60  minutes  -  90th  percentile  (Xq  ^0) 

Thus 

T  =*  30,  t  •»  60,  p0  ■  0.50#  p^  ■  O.65#  <Sq  *  0.05/  Q-  m  0.10. 

Sample  Size  and  Decision  Proceaure.  For  given  a  and  0  risks, 
the  determination  or  appropriate  sample  slse  and  critical  values 
is  a  relatively  complex  calculation.  Table  XXXVI  and  Table  XXXVII 
present  various  sampling  plans  for  c  -  0.10,  and  0.20,  values  of 
n  from  20  to  80  in  steps  of  10,  and  various  combinations  of  pQ, 

Qq,  Pp  qx. 

In  Table  XXXVI,  the  plans  are  such  that  the  median  and  90th 
or  95th  percentile  are  specified  In  the  Hq  hypothesis.  The  fol¬ 
lowing  four  combinations  are  included: 


Hq  Specification  Specification 


(1) 

0.50 

0,10 

0.65 

0.15  ] 

1  Median  and  90th 

(2) 

0.50 

0.10 

0.75 

0.20  J 

r  percentile  for  Hc 

■< 

(3) 

O.50 

0.05 

0.65 

0.10  1 

[  Median  and  95th 

(4) 

0.50 

0.05 

0.75 

0.15  J 

f  percentile  for  Hq 

In  Table  XXXVII,  the  planB  are  such  that  the  median  and  90th 
or  95th  percentile  are  specified  in  the  ft.  hypothesis.  The  fol¬ 
lowing  four  combinations  are  included:  A 


Hq  Specification  Specification 


(1) 

0.40 

0.05 

0.50 

0.10  1 

Median  and  90th 

(2) 

0.40 

0.025 

0.50 

C.10  J 

percentile  for 

(3) 

0.30 

0.025 

0.50 

0.05  1 

Median  and  95th 

(4) 

0.30 

0.01 

0.50 

0.05  J 

percentile  for 
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TABLE  Xmi 


SAMPLING  PLANS  FOR  Hq  SPECIFICATION 


Median  and  90th  Percentile 


Bor  Ha! 
Torn* 

»0 

Pi 

-  0.50 
«  0.65 

,  Oq  m  0.10 

<12  -  0.15 

For  Hq: 
For 

P0  -  0.50, 

PA  «  0.75, 

*0  " 
4l  - 

0.10 

0.20 

I  n 

c 

.  >■  0. 

10 

c 

*  0. 

20 

n 

c 

=  0 

10 

a 

=■  0.20 

j 

C 

D 

p 

C 

D 

& 

C 

D 

0 

c 

JO 

20 

13 

4 

0.52 

13 

3 

0.43 

20 

13 

4 

0.17 

13 

3 

0.00 

30 

20 

5 

0.50 

20 

4 

0.38 

30 

20 

5 

0.12 

20 

4 

0.08 

40 

30 

6 

0.58 

24 

6 

0.^3 

40 

30 

6 

0.20 

24 

6 

0.01 

50 

31 

8 

0.30 

29 

7 

0.13 

50 

31 

8 

0.02 

29 

7 

0.00 

60  j 

L_  37 

9 

0.24 

35 

8 

0.11 

60 

37 

9 

0.01 

35 

8 

0.00 

70 

44 

10 

0.25 

4i 

9 

0.09 

70 

44 

10 

0.01 

41 

9 

0.00 

80 

47 

12 

0,11 

48 

10 

0.10 

80 

47 

12 

0.00 

48 

10 

0.00 

■  -  _ 

_ 

Median  and  95th  Percentile 


Pbr  Hq:  pQ  -  0.50,  qQ  -  0.05  "For  HQ;  pQ  =  0.50,  qQ  =  0.05 

For  H^:  ?2  ”  0.65,  q^  ”  0<1°  For  %!  =  0.75,  q^  =  0.15 


n 

< 

X  -  0. 

10 

< 

x  *  0. 

20 

n 

a  -  0. 

10 

c 

=  0 

20 

c 

.  D 

e 

c 

0 

MM 

D 

e 

c 

D 

P 

20 

14 

2 

0.54 

12 

2 

20 

2 

0.19 

12 

2 

0.06 

30 

20 

3 

0,45 

21 

2 

30 

20 

3 

0.03 

21 

2 

0.07 

40 

25 

4 

0.31 

24 

3 

4o 

25 

4 

0.02 

24 

3 

0.01 

50 

30 

5 

0.20 

29 

4 

0.11 

50 

30 

5 

0.01 

29 

4 

0.00 

60 

37 

5 

0.18 

37 

4 

0.12 

60 

37 

5 

0.00 

37 

4 

0.00 

70 

42 

6 

0.13 

4l 

5 

0.07 

42 

6 

0.00 

4l 

5 

0.00 

80 

47 

7 

0.09 

45 

6 

0.03 

47 

7 

0.00 

45 

6 

0.00 

l68 


For  fy: 
For  Hjt 


cz 

1 

20 

c 

12 

30 

17 

4o 

21 

50 

25 

60 

31 

70 

35 

8c 

39 

tablb  mvii 

S««»LII!Q  FLAKS  FOR  SPBCOTCATIOif 


- -  WBdiyji  and  90th  FereantlU 

p0  ■  °-40'  %  •  0-05  1  For  tt. 

P1  "  °-50'  «1  *  0-W  Tor  H? 


P 

0.45 

0.39 

0.29 

0.2* 

0.21 

0.17 

0.13 


909  **>*  r0  "  °**°»  9q  “  0.025 

vor  %:  px  m  0.56,  qa  .  0ol0 


C 

D 

10 

1 

17 

,  i-k 

19 

2 

25 

2 

32 

2 

33 

3 

58  f 

"  3 

Por  P0  -  0.30,  -  0.01 

For  H^*  P2  *  0.50,  q  m  0 


0.09  19 

0.06  26 
0,l8  25 
0.02  30 


P 

0.21 

0.20 

0.06 

0.04 

0.10 

0.01 

0.01 


P 

0.34 

0.20 

0.11 

0.09 

0.03 

0.02 

C.01 


P 

0.24 

Q.07 

0.0- 

0.03 

0.02 

O.CI 

0.01 


mm 


‘  ■  „  ■  r  ^  : 


The  decision  procedure  is  baaed  on  two  critical  values,  C 
and  D.  An  accept  decision  is  cade  only  if  C  or  fewer  observed 
maintenance  times  are  greater  than  the  specified  median  value 
(T)  and  D  or  fewer  observed  maintenance  times  are  greater  than 
t,  the  time  corresponding  to  the  percentile. 

For  a  given  a,  a  plan  Is  identified  by  the  triplet  (n,  C, 

P) .  Tables  XXXVI  and  XXXVII  present,  for  each  n,  the  smallest 
value  of  C  and  the  corresponding  D  value  that  satisfy  the 
risk.  The  appropriate  plan  to  use  is  then  determined  from 
evaluation  of  the  0  risks  shown  in  Tables  XXXVI  and  XXXVII  for 
each  triplet.  In  general,  0  will  decrease  as  C  or  D  decreases 
with  n  fired,  or  0  will  decrease  as  n  increases  for  fixed  C. 

Since  n  was  limited  to.  only  seven  values  and  C  limited  to 
only  the  smallest  possible  value  for  a  given  a  risk,  a  desired 
0  risk  may  not  be  obtainable  from  the  tables  ( see  Discussion 
section) . 

Discussion.  This  plan  corresponds  closely  to  Test  Method 
No.  4',  KJL-§TB"471  (as  revised  by  Change  Notice  1).  In  method  4, 
however,  only  undesirable  (large  rejection  probability)  median 
and  H,  values  are  specified  and  no  consideration  is  given  to 

the  producer’ o  risk.  In  this  test,  both  producer  and  consumer 
risks  can  be  controlled. 

The  details  for  determining  producer  and  consumer  risks  for 
any  given  triplet  (n,  C,  D)  are  developed  in  Reference  6.  The 
equations  are  as  follows: 


Producer's  Risk  = 


>  -  £(;V '‘-o)-' f ©(£>  (■  -  g)’ \ 


J»0 

£ 


i 


-  /  f  7 — r-J ^ — —  U-p0)n‘J  (p0~<i0);,’k 

A  (n-j) !  k.'  (j-k)l  0  ^  0  ^ 


J-*0  k*0 

Consumer's  Risk  - 
C 


£(;y  »($(•-&) 


J-k 


1?0 


where  m  -  the  smaller  of  the  two  value o  j  and  D, 

The  O.C.  curve  of  a  plan  based  on  n,  C,  and  p  can  be  deve¬ 
loped  from  the  above  equation  for  consumer's  risk  lay  varying  the 
p  and  q  values.  Because  of  the  wide  variety  of  alternatives 
generated  by  the  triplets  (n,  C,  D),  such  evaluation  nay  fee  war¬ 
ranted  in  cases  where  careful  control  of  rftSH'-egsS  Wise 

is  desired. 

Examples.  Two  examples  are  given: 

Example  1  -  Assume  that  it  is  desired  to  accept  with  a 
90-percent  probability  an  equipment  with  a  median  value  of  20 
minutes  and  S5th  percentile  of  45  minutes.  If,  however,  20 
minutes  is  the  25th  per  entile  and  45  minutes  is  the  85th  per¬ 
centile,  only  a  10-percent  acceptance  probability  is  desired. 

In  terms  of  the  standard  hypothesis  for  this  test, 

Hq:  20  minutes  *  *0,509  45  minutes  ®  Xq  ^ 

H^:  20  minutes  »  xo#75*  ^5  minutes  »  Xq  ^ 

with  a  =  £  *  0.10.  We  thus  have  pQ  -  0.50,  q*  -  0.05,  pA  *>  0.75, 
*  0.15. 

Since  the  median  and  the  maximum  percentile  are  specified  in 
H0>  Table  XXXVI  is  appropriate.  For  the  specified  p,  q  values, 

the  plan  closest  to  meeting  .the  a  and  5  risks  is 


n  =  30,  C  -  20,  D  -  3 

Thus  30  maintenance  times  are  sampled.  The  equipment  passes 
the  test  only  If  20  or  fewer  such  times  are  more  than  20  minutes 
long  and  3  or  fewer  actions  took  over  45  minuses. 

Example  2  -  Assume  that  if  the  median  is  1  hour  and  the 
95th  percentile  Is  2  hours,  only  a  10-percent  acceptance  prob¬ 
ability  Is  desired.  However,  there  should  not  be  more  than  a 
20-peroent  rejection  probability  if  1  hour  is  the  70th  percentile 
and  2  hours  is  the  99th  percentile.  Thus 

Hq:  1  hour  *  Xq^q,  2  hours  -  Xq#01 
1  hour  «  Xq  2  hours  »  Xq 

with  a  ■  0.20  and  p  ■  0.10.  We  thus  have  F0  =  0.30,  qQ  =  0.01, 
p1  ®  0.50,  q^  *  0.05. 
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Since  the  median  end  maximum  percentile  values  are  specified 
in  the  R,  hypothesis.  Table  3007711  is  appropriate.  Prom  this 
table,  it  is  seen  that  the  specified  a  and  £  risks  are  satisfied 
if  n  -  30,  C  «  11,  D  -  1. 


7. 3.8.5 


-  Test  fpr,  gmlftQ  LfigQfiJOMtL 


Description  of  Test.  A  nonparametric  test  of  the  hypothesis 
that  maintenance  times  are  lognormally  distributed  with  parameters 
9  and  c 2.  The  Kolmogorov-Sairnov  statistic  Is  used  as  the  test 
statistic.  A  single  alternative  hypothesis  is  not  specified  and 
thus  no  Beta  risk  is  associated  with  h%  . 


Dh&erlylng  Assumptions.  The  distribution  of  maintenance 
v times  is  continuous. 

Hypothesis.  Let  F(x)  be  the  distribution  function  of  mainte¬ 
nance timesVire . ,  F(x)  -  P(X  s  x)  and  FL  (X;  e,  ©2),  the  distri¬ 
bution  function  of  a  log-normal  with  parameters  9  and  .  Then 

ty  F(jc)  -  F0(x)  -  FL(xi  90,  0%) 

H^:  F(x)  ■  P^(x)  t  Fq(x)  for  all  x 

✓  Table  XXX U  of  this  subsection  should  be  used  to  find  and 

2  *  ^ 
a  q  if  another  pair  of  lognormal  parameters  (e.g.,  mean  and  per¬ 
centile}  is  specified. 

The  graphic  illustration  of  the  two  hypothesized  distribution 
functions  is  shown  below.  For  any  value  of  x,  the  probability  of 
completing  a  maintenance  action  is  greater  under  Hq  than  under  H-. 

1.0, 


F(X)  -P[X«x) 


0 


Time,  x 


Sample  Slse.  Ho  specific  equation  for  cample  else  can  be 
given .  The  greater  n  la#  however#  the  more  powerful  the  fc eat. 

will  be  against  any  specific  alternative  hypothesis. 

-**■*— ^  < —  '  ^  .  * 

Decision  Criterion.  A  random  sample  of  n  maintenance  times 
ls  observed.  It  is  necessary  to  compute  the  statistic  D,  which 
is  defined  by  the  equation 

®  -  T  [h  (*J  »o'  °so>  -  SH 

where  (x;  0Q,  a2^)  Is  the  hypothesised  lognormal  distribution 
function  and  S(x)  Is  the  observed  distribution  function. 

To  be  able  to  use  published  tables  of  the  D  statistic#  it  is 
necessary  first  to  consider  the  variable  Y.  ~  «  inX.  where  X,  is 

4>t>  1  1  «*i» 

the  1  -ordered  observed  maintenance  time,  l.e.,  3^  £  Xg  £  X^  s£ 

. . .  *  s  Then  the  statistic  to  be  computed  Is 

3(y)  -  fn  (y;  oQ,  c20)l 

where  Fn  (y;  QQ,  is  the  normal  distribution  function  with 

mean  0Q  and  variance  a2Q. 

The  procedure  is  as  follows: 

(1)  Let  Y=-inX  for  each  of  the  observed  n  times  X^,  Xg# 

(2)  Order  the  Y's  such  that  s  Yp  s  . . .  i  ^n-l  s  ^n 

(3)  Compute 

S(y  )  »  number  of  Y's  leas  than  or  equal  to 

(4)  Compute  for  each  1  the  normal  deviate. 


max 

y 


173 


TV  » 


M  For  each  ZjL,  use  normal  probability  tables  to  obtain  the 
normal  probability 

PN  (”Yi;  V  °2 o')  "  fn  (Zi;  °»  1) 

which  la  equal  to  the  probability  associated  with  a  normal  deviate 

(6)  Let  0H  (-Yi;  90,  o2q)  -  1  -  PN  (-Yi;  80<  o20) 

(7)  For  each  Y^,  compute 


7^  •  °H  (Y1;  ®0*  “Vj 


as  follows: 

For  i«l,  compute  S  (Y1)  -  GN  (Y1) 

For  1  <  i  s  n,  compute 
s<Yi-l)  ‘  MYi)  8,56  s(Yi)  " 

8  then  equal  to  the  largest  of  the  above  differences. 

^  a  risk,  refer  to  a  table  of  the  Kolmogorov- 

Sa:imov  statistic  for  the  critical  D  value  D 

*  a,n* 

Accept  IL  if  B  s  D 

v  a,n 

Reject  Hq  otherwise. 

is?  xs^MitrsssatM  bybH*)“^>-otlon  18 10  the  rlght  of 


i?u 


; 

’■&  •*>  J  '  T 


*  ;  .  , 


•Jf^X 

'  X.  ;•=??■ 


'  *>-Y 


't  v 

v,  '  '.IK 


Specified  Fq(x) 


F(X)  «  pfX  sx] 


"“-Observed,  S(x) 


Time,  x 

then  the  data  would  tend  to  favor  and  thus  ths  hypothesis  would 

be  rejected.  In  this  case,  the  statistic  S(x)  -  FQ(:c)  Is  small 

(in  fact,  it  is  negative).  So  that  tabled  values  of  the  D  sta¬ 
tistic  can  be  used,  the  conversion  Is  made  to  negative  values  of 
the  observed  maintenance  times  and  [1  -  Fq(-x)I  la  used  with  the 

result  that  small  values  of  [S(x)  -  FQ(x)]  correspond  to  large 

values  of  [S(-x)  -  (I-Fq(-x)]. 

( 

The  conversion  to  logarithms  is  first  made  solely  to  that  it 
will  be  possible  to  use  the  normal  distribution  tables  rather  than 
have  to  develop  tables  of  the  lognormal-distribution  function. 

Example .  Assume  the  following: 

ilQ  is  F(x)  -  Fl(x;  6=3,  a2  =  0.25). 

Hx  is  F(x)  <  Fl(x;  6=3,  o2  =  0.25). 

a  is  set  equal  to  0.10. 

Assume  that  a  sample  of  10  observations  yields  the  following 
maintenance  times  (minutes):  3.5.4,  14.8,  30.1,  35.6,  12.7,  14.8, 
24.8,  63.4,  13.0,  and  38,4.  The  table  at  the  top  of  the  following 
page  is  developed  in  accordance  with  steps  1  through  6. 
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D 

Steps  1  and  2 
Y±*  -inXi 

Step  3 
8(5^) 

Step  4 

-v6 

Zi”  0 

Step  5 

w 

Step  6 

°N<yi>=1-?N<7i> 

63.* 

-4.14 

0,1 

0.99 

0.01 

38.4 

“3.65 

0.2 

0.90 

0.1C 

35.6 

-3.57 

0.3 

1.14 

0.87 

0,13 

30.1 

-3.40 

0,4 

o.8o 

0.79 

0.21 

24.8 

-3.21 

0.5 

0.42 

0.66 

0.34 

15.4 

-2.73 

0.6 

-0.54 

0.30 

0.70 

14.8 

-2.69 

0.8 

-0.62 

0.27 

0.73 

14.3 

-2.69 

0.8 

-0.62 

0.27 

0.73 

13.0 

-2.56 

0.9 

-0.88 

0.19 

0.81 

12.7 

-2.54 

1.0 

-0.92 

0.18 

0.82 

Then  for  Step  7 >  the  following  are  obtained: 


3<Yi-l>  -  °n'Yl> 


s(*i)  -  an(Y1) 


1 

— 

0.09 

2 

-  0 

0.10 

3 

0.07 

0.17 

4 

0.09 

0.19 

5 

0.06 

0.16 

6 

-,1 

-0.20 

-0.10 

l) 

-0.13 

0.07 

9. 

-0.01 

0.09 

•10 

0.08 

0.18 

Since  the  maximum  difference  is  0.19#  which  is 
critical  value  of  0.32  for  a  sample  size  of  1.0,  arid 
null  hypothesis  of  a  lognormal  distribution  with  9  = 

.o.  w 

<£  = 


0.25  oannot  be  rejected. 


7.3.8. 6  Teet^ Number  -  Sequential  Test  of  a  Critical 


itenance  Time 


prob 


„  of  Test.  A  nonparametric  sequential  test  of  the 
or  exceeding  a  specified  maintenance  time.  Since  the 
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procedure  is  based  solely  or.  the  number  of  maintenance  times 
exceeding  the  critical  time,  it  is  equivalent  to  a  sequential 
test  of  a  binomial  parameter,  p. 


Underlying  Assumptions .  No  specific  form  of  the  distribution 
of  maintenance  times  is  assumed.  Maintenance  tasks  are  sampled 
through  a  simple  random  procedure. 


Hypotheses. 

Ho:  T  =  \ 

or 

P  [  X  >  T  ]  -  pQ 

H, :  T  =  X 

1  Pj 

or 

P  [X  >  T]  =  Pl 

(a  and  8 

are 

specified) 

Sample  Size.  The  sample  size  of  a  sequential  test  is  a 
random  variable.  Table  XXXVIII  presents  the  expected  number  of 
observations  before  a  decision  for  three  values  of  p. 


TABLE  XXXVIII 

EXPECTED  SAMPLE  SIZES  FOR  VALUES  OF  p 


P 

E(nip) 

p0 

(1-a)  in  B  +  a  inA 

P0inC  +  (l-p0)inD 

inD 

in  A  in  B 

inD-  inC 

in  C  in  D 

P1 

.  piiffl  +  fl --9)  in  A 

Pl  in  C  +  (l~Pl)  in  D 

Notation:  A  = 

(l-8)/a,  b  =  8/(1 -a) 

C  = 

(p/Pq),  D  =  C  (l-Pl)/(l-p0)  ] 

If  desired,  an  upper  bound  on  the  sample  size  can  be  estab¬ 
lished  that  will  have  limited  effects  on  the  a  and  8  risks.  The 
suggested  procedure  is  to  truncate  tbe  test  at  three  times  the 
expected  sample  size,  say  at  m=n* ,  The  expected  sample  size  to 
use  will  depend  on  the  assumed  value  of  p  in  the  expression 


Generally,  pQ  is  select «»d  unless  prior  evidence  indicates 
otherwise j  therefore, 


Decision  Procedure.  The  decision  procedure  is  as  follows 


(1)  Compute  A 


(2)  Compute  C  =  in 


)  Compute  the  acceptance  boundary 


(4)  Compute  the  rejection  boundary 


Random  samples  cf  maintenance  times  X. 
obtained  as  long  as 


are  then 


where  d^  is  the  nuraoer  of  maintenance  times  that  exceed  T  after 

m  observations  are  made.  A  decision  is  made  the  first  t.’ve  the 
above  Inequality  is  violated: 


Accept  H0  if  for  some  m,  d  & 


If  for  some  m,  d  &  b 


If  the  truncated  sample  size  (say,  n*) 
decision  has  been  made; 


reached  and  no 


Accept  Hn  if  d  -  < 


Reject  Hq  If  dn#  > 


' — r  . 


Discussion.  This  test  and  Test  Method  1  of  MIL- STD-471  are 
similar  in  that  both  are  sequential  tests  of  percentile  values. 
However,  the  MIL-STD-471  plan  assumes  a  mean  or  specification 

under  a  lognormal  assumption,  which  is  then  converted  to  specifi¬ 
cations  on  percentiles. 

This  test  is  a  standard  application  of  sequential  sampling 
under  a  binomial  assumption.  Reference  8,  pp.  88-105,  presents 
a  detailed  discussion  of  the  test's  operating  characteristics  as 
well  as  the  practical  consequences  of  talcing  observations  in 
groups . 

7 -3.8. 7  Teat  Number  14  -  Sequential  Test  of  Two  Critical 
Maintenance  Time's 

Description  of  Test .  This  test  is  the  sequential  counter¬ 
part  of  Test  dumber  ll.  Two  critical  maintenance  times  are 
specified,  generally  a  median  and  value'.  The  decision 

criterion  is  developed  from  a  direct  application  of  the  sequen¬ 
tial  proDabllity-ratio  test. 

Underlying  Assumptions.  No  specific  assumptions  concerning 
the  distribution  of  maintenance  times  are  necessary.  Simple 
random  sampling  is  performed. 

Hypotheses . 

Hq:  T  -  100 


t  =  100 


Hxs  T  =  100 


t  «  100 


where 

T  <  t,  pQ  <  px,  qQ  <  q.,  p±  >  qi  for  1  =  0  or  1 

Mote  that  p^  Is  the  percent  of  observations  greater  than  T  for 
ri, ,  and  la  the  percent  of  observations  greater  than  t  for  H^. 
Risks  of  ct  end  £  are  also  specified. 


x  (l-p0)percent:T  e  = 


x  (l-q^percentlle.  =  Xc 


«o 

x  (l-q^)percentile  =  X 


x  (1-p^percentlle  - 


Qi 
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Sample  Size.  The  sample  size  of  a  sequential  test  is  a 
rsndoivwiableT  For  the  e&aes  in  which  HQ  and  ^  are  true, 

the  following  are  the  equations  for  the  expected  sample  size: 


E(n|HQ  true) 


( 1-a) inB  +  a  ink 

Ti-p0)<3  +  (Pq-Qq^d  +  q^E 


EfnjH^  true) 


SinB  +  (l-fl)inA 

+Ip^-qr)D  + 


where  A,  3,  C.  D,  end  E  are  as  defined  below. 

Decision  Procedure. 

(1)  Compute  A  -  ® 

(2)  Compute  C  -  1«(^)  D  -  ln(^)  E  =  ,.^1) 

(3)  Compute  the  acceptance  boundary 

a  *  -inB  +  mE 
m 

(4)  Compute  the  rejection  boundary 

bm  =  -in A  +  mE 
m 

Random  samples  of  maintenance  time3  Xlf  Xp,  ...  are 
then  obtained  as  long  as  1  d 

b  <  (D-C)N  (T)  +  (E-D)N(t)  <  a 
m  m  m  m 

where 

N  (T)  =>  number  of  maintenance  times  less  than  or 
“  equal  to  T  after  m  observations 

N  (t)  =  number  of  maintenance  times  less  than  or 
equal  to  c  after  m  observations 

A  decision  is  made  the  first  time  the  above  inequality 
is  not  true.  Thus 

Accept  Hq  if  for  some  m,  (D“C)Nm(T)  +  (E-DjN^t)  % 
Reject  Hq  if  for  some  m,  (D-C)Nm(T)  +  (E-D)Nm(t)  *  bm 
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Discussion,  This  test  Is  comparable  to  Test  Method  1  of 
MIL-3td-4ti  ir>  that  both  are  sequential  tests  of  two  percentile 
values.  The  MIL- STD-472  test,  however,  is  based  on  a  conversion 
of  a  lognormal  me an/pere entile  specification  to  a  two-percentile 
specification;  fo:*  each  such  specification  a  separate  sequential 
decision  criterion  is  then  applied.  The  MIL- STD-471  method 
appears  to  be  based  on  an  approximation  method  for  using  the 
same  data  for  the  two  separate  tests  in  order  to  satisfy  the 
alpha  and  beta  risks. 

In  this  test,  only  one  sequential  decision  criterion  is 
applied;  it  takes  into  account  the  dependence  between  the  number 
of  observations  less  than  or  equal  to  T,  the  number  between  T 
and  t,  and  the  number  greater  than  t.  The  test  is  therefore 
exact  in  the  sense  that  any  sequential  probability-ratio  test  is 
exact .  . , 

If  desired.  Table  XXXII  can  be  used  to  convert  a  lognormal 
mean/percentile  specification  (or  any  other  lognormal  parameter 
pair)  to  a  specification  of  two  critical  maintenance  times  so 
that  this  test  will  be  applicable .  If  the  lognormal  assumption 
can  be  reasonably  made,  however,  the  parametric  counterpart  to 
this  test  (Test  Number  6)  will  generally  be  the  more  desirable 
choice . 

Since  this  test  does  not  generally  appear  in  the  literature, 
its  derivation  is  outlined  below. 


The  basic  inequality  for  the  sequential  probability-ratio 
test  is 

m 


Ihj  *  B  < 


£,(W  n, 

m  Prvn 

1“1  ffXjHo)  0m 


<  A  = 


where  f(Xi|Hj)  is  che  density  of  the  ir  1  observation  under  H^, 

j  =  0,  1. 

Plw. 

If  - — 1  <5  A,  then  the  data  indicate  that  H-,  is  significantly 
p0m 

more  consistent  with  the  observations  than  HQ,  wlth^the  result 
that  Hq  is  rejected.  Similarly,  Hq  is  accepted  if  s  B.  For 

this  problem  the  observed  maintenance  tiraeo  can  be  classified  as 
being  less  than  or  equal  to  T  or  less  than  or  equal  to  t,  gen¬ 
erating  the  two  random  variables  Nm(T)  and  N^t)  as  defined 

above .  Then  it  can  be  shown  that 


3  -  0,  1 


P*  "  PW,(T)  -  k0,  M„(t)  -  K,) 

a!  .“o,  ,kl'k0  ■-ki 

‘  H^nqiTn-T^TT  (1'p3)  (p3'V  q3  * 

which  is  a  trinomial  probability  function  (the  third  variable  is 
the  number  of  observations  greater  than  t,  which  is  always  equal 
to  m-kj).  When  the  above  expressions  for  and  pln  are  inserted 

into  the  basic  inequality,  the  result  after  taking  logarithms 
and  simplifying  is  as  given  above  in  the  Decision  Procedure  sec¬ 
tion. 


is 


For  the  expected  sample-size  equations,  the  basic  formula 


EfnjHj) 


L(H^)inB  +  [l-L(H4)JinA 


E 


r^i^n 

[*n?TY|H^7_ 


where 


where 


L(Hj)  »=  P [accept  HQjHjtrue]  (L(Hq)  =  l-o,  L^)  *  p) 

Ej  is  the  expectation  operator  und.,.'  the  condition  true 

Since  f(X|Hj)  in  this  problem  is  equal  to  ffN^T),  N^vtJjH^, 

then  N,(T)  [N, (t)-N, (T )  ] 

N1(t'l|HJ)  -  (1-Pj)  1  (Pj-qj)  1  1 

[l-N-(t) J 


for  J  -  0,  1 


(T)  .  (o  If  X  >  T 
1  ' 1  if  X  S  T 


if  X  >  t 
if  X  S  t 


-  f(X  JH,  )-j  /L-p,\  /p,-q,v 

E3[innwj  -  +  (pj-q3u<v^) + 

qj  for  3  -  0,  1 

which  la  the  denominator  of  the  sample-size  equation 
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Example .  Assume  the  following  set  of  hypotheses  for  a  test 
for  which  a  *  p  =  0.10. 

Hq:  0.25  hrs.  -  median  «  50th  percentile,  1  hr.  -  95th 

percentile 

O.25  hra.  =  35^h  percentile,  1  hr.  ■  $0^  percentile 

In  terms  of  the  notation  used,  the  following  is  equivalent: 

K0:  0.25  *  T  *  XQ  ^0,  1.0  -  t  - 

H1‘*  0,25  “  T  “  X0.65>  1,0  *  t  “  ^.lO 

Hence 

T  =  0.25,  c  *  1.0,  pQ  *  0.50,  q^  -  0.05,  P-^  -  O.65,  q.^  » 

0.10 

Then 

ink  =  in9  =  2,1972,  inB  *  in(l/9)  «  -2.1972 
C  =  Hrlo  =  "°*3567,  D  =  |n£^j|  «  0.2007 
E  =  »  0.69315 

The  acceptance  boundary  is  «  2.1972  +  0.6932m 
The  rejection  boundary  is  bm  *  -2.1972  +  0.6932m 
The  test  statistic  5s 

\(T)(D-C)  +  Nm(t)(E-D)  «  0.5567Nm(T)  +  0.4925Nm(t) 

where 

N  (T)  Is  the  cumulative  number  of  observations  less  than 
m 

or  equal  to  T  =  0.25  hrs. 

Nm(t)  Is  the  cumulative  number  of  observations  leas  than 
or  equal  to  t  =  1.0  hrs. 


For  the  expected  staple  sizes*  E(n|R’0)  «  34  and  E{n|Hi)  »  33. 

These  expected  values  correspond  to  a  fixed -sample- size  require¬ 
ment  of  almost  80  observations  (see  Table  XXXVI). 

If  the  hypotheses  of  the  example  of  Test  Number  6  are  con¬ 
verted  to  two  critical  maintenance  times  so  that  Test  Number  14 
can  be  applied,  the  expected  saaple  sizes  are  35  if  Hq  is  true 

and  32  if  is  true.  These  values  correspond  to  the  expected 

sasple  sizes  of  the  Test  Number  6  application  of  29  and  26, 
respectively,  indicating  the  greater  efficiency  of  the  parametric 
test. 

7.4  GUIDELINES  FOR  TEST  SELECTION  -  NON- BAYESIAN  TESTS 

7.4.1  Introduction 

Fourteen  different  non-Bayesian  tests  that  can  be  used  for 
maintainability  demonstration  have  been  presented  in  Subsection 
7.3.  In  this  subsection,  guidelines  are  presented  for  selecting 
the  test  that  is  appropriate  for  a  particular  situation. 

7.4.2  Summary  of  the  Fourteen  Teats 

Generally,  the  factors  associated  with  the  maintainability- 
demonstration  program  will  restrict  the  choice  of  test  to  one  of 
two  alternatives.  Table  XXXIX  summarizes  the  fourteen  tests  with 
respect  to  ten  major  factors  that  are  relevant  to  the  choice  of 
method. 

7.4.3  Declaion  Tree  for  Selecting  a  Test 

Table  XL  is  a  decision  tree  derived  from  Table  XXXIX;  It 
indicates  which  teat  will  me^t  requirements  on  type  of  sampling, 
distribution  assumption,  and  parameter  specification.  Thus,  a 
fixed-sample  test  of  the  median  based  on  a  lognormal  assumption 
should  be  based  on  Test  2. 

Several  alternatives  exist  in  the  tree.  For  example,  both 
tests  3  and  4  are  percentile  tests  based  on  a  fixed  sample  size 
and  lognormal  assumption.  Reference  to  Table  XL  or  Subsections 

7. 3. 6. 4  and  7. 3-6. 5  reveals  that  test  3  is  a  test  of  a  critical 
percentile  (p  is  fixed),  while  test  4  is  actually  a  test  of  a 
critical  maintenance  time  (T  is  fixed),  although  both  have  a  per¬ 
centile  specification  for  the  null  hypothesis. 

The  distinction  between  the  nonparametric  tests  9  and  10  is 
similar.  Sequent ial  test  6  differs  from  tests  7  and  8  in  that  it 
is  based  on  a  Joint  <aoecification.  Tests  7  and  8  differ  in  the 
sem*  sense  as  tests  3  uad  4.  Test  11  differs  from  test  12:  In  that 
the  former  is  a  test  of  a  median  and  specification  without 

regard  to  distributional  form, • while  the  latter  is  a  test  for  a 
specific  lognormal  distribution,  which  is  defined  by  the  pair  of 
specified  parameters. 
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SUMMARY  OF  FOURTEEN  TEST  PLANS  WITH  RESPECT  TO  TEN  MAJOR  FACTORS 


valuo  of  can  b«  u mrl  aa  a  gatvaraJ  r-jlc  of  thumb. 


It  is  still  necessary  to  choose  between  fixed  or  sequential 
and  parametric  (lognormal)  or  nonparametric  tests.  These  alter¬ 
natives  have  been  discussed  in  Section  IV. 


separate  s&nples  are  to  be  used  for  each  Individual  test,  the 
individual  test  risks  can  be  determined  from  overall  risks  as 
follows:, 


Let  7  and  V  be  the  parameters  tested  by  the  two  tests,  with 
7q  and  V'q  representing  desirable  levels  and  7-j^  and  ^  representing 
undesirable  levels.  The  overall  test  risks  are  defined  as  follows 

o  =  Pfreject  if  7  =  7Q  arid  V  =  ^03 

fi  =  P[accept  if  7  -  7-^  or  1*  =  ^3 

Then,  if  t*i  and  fn  represent  the  test  risks  for  the  1th  test 
(i=l  or  2),  under  the"  assumption  of  independence,  the  following 
rule  should  be  observed: 

Choose  aj  and  a 2  such  that 

(l~ai)  (1“a2^  =  1  ’  a 


and 


choose  Pi  and  P2  such  that 
Pj  +  ^2  “  ^1^2  *  P 

This  rule  allows  for  assigning  importance  factors  for  one 
test  over  the  other.  If  both  tests  are  considered  equally  im¬ 
portant,  the  results  ere  as  follows: 

jL 

(*i  =  a2  =  1  -  (l"0t)? 

Pi  *  pg  =  i  -  (i-e)* 


7-5  BAYESIAN  TESTS 
7.5.1  General 


Tests  for  which  prior  information  is  incorporated  In  the 
decision  criterion  through  use  of  Bayes’  Theorem  are  called 
Bayesian  tests.  If  p  is  the  parameter  of  interest,  prior  in¬ 
formation  on  p  is  available  in  the  form  of  a  density  function 
g(p),  and  n  sample  observations  =  (Xx,  x?,  ...,  x  )  are  made, 

Bayes’  Theorem  leads  to  the  following  equality: 
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f(plXn) 


L(Xnlp)  g(p) 


p)  s(p) 


dp 


where  fCpIX^)  is  the  posterior  density  of  p  after  the  sample 
observations  2Ln  are  observed. 

^(XjJp)  ia  the  likelihood  of  X  given  p.  Since  p  is  inte¬ 
grated  out  in  the  denominator,  the  above  equation  can  be  rewritten 
as 

fCpIX^)  =  i  L(xjp)  g(p) 


where  K  is  a  proportionality  constant.  This  equation  indicates 
that  the  posterior  distribution  of  p  is  equal  to  the  prior  dis¬ 
tribution  modified  by  t^e  observed  test  results  5^. 

A  Bayesian  test  can  be  based  on  the  properties  of  the  pos¬ 
terior  distribution.  For  example,  the  posterior  mean  value  of 
the  parameter  p  can  be  computed  from  the  equation 

EfpIXj  -  f  P  f(pl dP 
-  00 

and  the  accept/reject  decision  made  according  to  whether  or  not 
E(plX^)  falls  within  a  desirable  region. 

There  are  several  advantages  of  a  Bayesian  test  in  maintain¬ 
ability  demonstration  over  so-called  classical  procedures.  If 
information  is  available  on  the  maintainability  characteristics 
of  an  equipment,  then  a  decision  procedure  employing  that  infor¬ 
mation  in  the  form  of  g(p)  plus  the  additional  test  information 
is  obviously  more  complete  than  one  based  solely  on  X^. 

Secondly,  the  practical  consequences  of  a  Bayesian  test  can  be 
important.  If  the  prior  density  is  such  that  there  i3  high 
assurance  that  the  equipment  is  satisfactory,  a  Bayesian  test 
will  generally  require  relatively  little  additional  testing.  On 
the  other  hand,  if  g(  p)  is  unsatisfactory,  the  product  can  be 
accepted  only  after  relatively  extensive  testing. 
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Thirdly,  a  Bayesian  test  can  be  constructed  to  provide  a 
control  that  may  be  more  pertinent  to  the  needs  of  the  customer. 

A  classical  test  essentially  controls  the  acceptance  probability 
for  specified  levels  of  desirable  and  undesirable  product.  Con¬ 
sidering  the  latter,  for  example,  the  Beta  risk  control  is  defined 
as 


P  [Accept  I  Maintainability  Is  unacceptable]  =  {3 

A  Bayesian  test,  on  the  other  nand,  can  be  designed  to  control  the 
maintainability  of  the  accepted  equipment  by  the  criterion 

P[  Maintainability  is  unacceptable  I  accept]  -  p 

A  specific  Bayesian  maintainability- -demonstration  test  that 
is  based  on  a  control  on  accepted  product  maintainability  will  be 
presented  in  this  subsection.  Because  of  the  newness  of  tills  type 
of  test  and  its  broad  implications,  the  condensed  format  used  for 
describing  the  nonsequential  tests  will  be  replaced  by  a  more 
detailed  description  including  mathematical  derivation  .1  the 
test  criterion. 

7.5.2  Basic  Assumptions 


The  basic  assumptions  are  as  follows: 

(1)  The  maintenance-time  random  variable.  X,  has  a  lognormal 
distribution  with  parameters  0  and  a2  where  0  -  E(/nX) 
and  cj2  =  Var(fnX). 

(2)  The  parameter  0  has  a  normal  prior  distribution  with 
mean  (3  and  \ariance  w2. 

(3)  The  parameter  cr  is  known  or  can  be  accurately  estimated. 

Assumption  1  is  the  usual  application  of  the  lognormal  dis¬ 
tribution  for  maintenance- time  description.  Data  collected  end 
analyzed  by  ARINC  Research  on  this  and  other  studies  strongly 
support  this  assumption.  Further  support  for  use  of  the  lognormal 
distribution  for  maintenance-time  analyses  is  provided  in  many 
other  studies  involving  analysis  of  observed  maintenance  times. 

Assumption  2  Is  made  for  three  reasons.  First,  the  use  of  a 
normal  prior  distribution  for  0  allows  relatively  easy  develop¬ 
ment  of  a  Bayesian  test  since  It  leads  to  a  posterior  distribution 
that  is  also  a  normal.  (Priors  that  lead  to  .0  posterior  density 
of  the  same  form  are  called  conjugate.)  Secondly,  the  normal 
distribution  j  symmetrical,  characteristic  and  its  other  well  known 
properties  permits  the  use  of  easy  and  known  methods  of  quantify¬ 
ing  * ts  parameters  either  from  subjective  evaluation.?  or  observer 
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data.  Third,  the  fact  that  so  many  real  variables  of  measurement 
can  be  adequately  described  by  a  normal  distribution  attests  to 
its  versatility.  In  the  absence  of  data  supporting  a  nonsymmstri- 
cal  prior,  the  normal  seems  as  reasonable  a  choice  as  any. 

The  Bayesian  test  to  be  described  does  not  depend  or  a  norm?! 
prior  for  its  theoretical  justification.  Nonsymmetrical  priors 
or  discrete-type  distributions  can  be  employed  once  suitable  modi¬ 
fications  are  made. 

Assumption  3  is  made  primarily  for  simolification  in  develop¬ 
ment  of  the  procedure.  Otherwise,  a  prior  density  for  c2  would 
also  have  to  be  employed,  leading  to  quite  complex  mathematical 
statistics  involving  joint  densities.  Reference  9,  pp.  298-309, 
discusses  this  type  of  case.  Also,  under  Assumption  3,  the 
specified  index  of  maintainability  can  be  either  the  mean  or  a 
percentile  value,  because  if  a2  is  known,  a  mean  or  percentile  speci¬ 
fication  can  be  translated  into  a  specification  on  0.  If  the 
median  ftf  is  specified,  e  is  directly  determinable  from  the  median 
independently  of  a2. 

In  Subsection  7.5. 9>  the  date,  collected  in  this  study  are 
analyzed  and  prior  distributions  for  avionic  and  ground  equip¬ 
ments  are  developed  which,  in  lieu  of  more  applicable  procedures, 
can  be  used  to  satisfy  Assumption  2.  Furthermore,  several  methods 
for  using  predictions,  data,  and  subjective  evaluations  for  estab¬ 
lishing  a  prior  distribution  for  9  are  discussed.  To  satisfy  As¬ 
sumption  3y  the  data  and  prediction  equations  presented  in  Sub¬ 
section  7.3.5  car.  be  used  to  provide  an  estimate  of  o2. 

7.5.3  Maintainability  Index  and  Test  Requirements 

The  index  of  maintainability  that  can  be  used  for  the  test 
is  the  mean,  the  median,  or  a  percentile  value.  It  is  first 
necessary,  however,  to  translate  requirements  based  on  one  of  the 
above  indices  to  equivalent  requirements  on  9. 

As  for  conventional  tests,  two  levels  of  maintainability  are 
to  be  specified  —  a  desirable  level  (to  be  denoted  by  the  sub¬ 
script  0)  and  a  minimum  acceptable  cr  undesirable  level  (to  be 
denoted  by  the  subscript  1).  T.ble  XLI  shows  the  conversion  of 
mean,  median,  and  percentile  specifications  to  equivalent  speci¬ 
fications  on  9,  assuming  that  a2  is  known. 

Given  that  two  values  of  9 ,  0^  and  (@1  >  9Q)  are  speci¬ 
fied,  the  requirements  on  $he  3ayesian  test  that  are  considered 
here  are  as  follows: 

•  Requirement  I:  If  9  S  0n,  the  probability  of  passing  the 

test  is  uhigh. 

*  Requirement  II':  If  an  equipment  passes  the  test,  the  prob¬ 

ability  that  9  is  greater  than  is  low. 
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TABLE  XL I 

CONVERSION  OF  MEAN,  MEDIAN,  AND  PERCENTILE  SPECIFICATIONS 
TO  0  SPECIFICATIONS  --  a?  KNOWN 


Specified  Values 

Equivalent  Specifications 

Mean  |1q 

6q  =  in  -  aV2 

0 1  =  in  |ii  -  a2/2 

Median  SL 

0o  «  in  0O 

Mi 

@1  =  in  Mx 

Fth  Percentile  X  =  Tn 

P  o 

eo  -  To  “  zPa 

xp  =  ti 

0,  =  in  T,  -  Z  a 

1  1  P 

These  two  test  requirements  can  be  stated  more  precisely  a3 
follows:  Let  Tfi  be  some  calculated  statistic  based  on  a  sample 

of  n  observations  of  maintenance  time.  Let  T*  be  some  preselected 
critical  value  for  decision  such  that  the  equipment  passes  if 
Tn  £  T*  and  falls  if  >  T*.  Then- the  above  two  requirements 

can  be  written  as 


p[Tn  siT*le  =  e0]  =  i-  a  (1) 

P[e  >  0-jJTh  £  T*]  S  Pb  (2) 

If  it  is  assumed  that  a  and  Pb  are  specified,  the  only  un¬ 
knowns  in  the  above  two  equations  are  n,  the  sample  size,  and  T* 
the  critical  value  for  the  statistic  Tn>  The  objective  of  the 

Bayesian  analysis  is  to  determine  these  two  values. 

The  requirement  expressed  by  Equation  1  is  the  usual  pro¬ 
ducer's  risk  control,  whereby  there  Is  a  high  probability  of 
acceptance  if  the  mean  maintenance  time  or  man-hour  value  Is  at  { 
the  desirable  level. 

The  requirement  expressed  by  Equation  2  offers  the  consumer 
assurance  that  the  maintainability  level  of  accepted  equipment  at 
least  meets  a  minimum  requirement.  The  notation  £b  Is  used  to 

distinguish  the  Bayesian  risk  from  the  classical  Beta  risk.  In 


191 


actuality,  Equation  2  represents  a  control  on  the  upper  (l-p^) 
percentile  of  the  posterior  distribution  of  the  parameter  B  . 

A  test  based  on  the  requirements  I  and  II  is  believed  to 
represent  the  viewpoints  of  the  producer  and  consumer  better  than 
the  conventional  test.  The  producer,  in  his  own  best  interest, 
will  attempt  to  provide  equipment  that  equals  or  betters  the  0^ 

value,  but  he  would  like  high  assurance  that  if  he  does,  nls  equip¬ 
ment  will  not  be  rejected.  This  assurance  is  provided  by  require¬ 
ment  I  in  the  same  manner  as  conventional  procedures. 

The  consumer's  best  interests  are  served  by  the  more  direct 
approach  of  assuring  acquisition  of  satisfactory  products  (in  a 
distributional  sense)  rather  than  controlling  the  probability  of 
accepting  poor  product.  This  direct  control  is  provided  by  re¬ 
quirement  II. 

7.5.4  General  Bayesian  Formulation  of  the  Test 


Let  0  be  a  parameter  of  a  probability  density  function  that 
has  a  prior  density  g(e).  Let  Tn  be  a  statistic  based  on.  n 

observations  whose  distribution  depends  on  0.  From  Bayes'  Theorem, 
the  posterior  density  of  0  is 

f(e|Tn)  =  £  L(Tn|0)  g(0)  (3) 


K  =  f  L(Tn|e)  g(0)  d0 


L(Tn  1 0 )  =  likelihood  of  Tn  given**) 

Equation  3  provides  a  means  for  evaluating  the  distribution 
of- 8  given  the  statistic  Tn*  In  designing  an  accept/reject  demon¬ 
stration  test  that  is  to  meet  requirements  of  the  form  represented 
by  Equation  2,  the  sample  size,  n,  and  the  critical  value  of  Tn, 

say  T*,  are  to  be  determined  beforehand  on  the  basis  of  the  know¬ 
ledge  that  the  decision  process  will  be  such  that  Tn  s  T*  will  lead 

to  acceptance  and  Tn  >  T*  will  lead  to  rejection.  This  type  of 

Bayesian  consideration  involving  future  decision  actions  based  on 
the  results  of  testing  is  a  form  of  preposterior  analysis  as 
defined  in  Reference  9,  page  70. 

Prom  this  viewpoint,  the  "given"  portion  of  the  posterior 
density  of  8  can  be  extended  to  be  the  information  Tn  si  T*,  and 

the  posterior  density  to  consider  Is  defined  to  be 
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■p-sr 

f(e'Tn  *  TO  =  z  J  L(Tnj  0';  g(0)  dT  j 


!  I 


0 


where 


r00 

X  =  J  J  L(Tn|0)  g(0)  dTnd© 


-OO  “00 

7.5-5  Derivation  of  the  Posterior  Density 


In  this  subsection^  a  closed-form  expression  for  f(elTnsT*) 

is  derived  under  the  assumptions  listed  above.  Because  of  the 
normality  of  -^nX,  the  natural  statistic  on  a  test  of  q  ,  the  ex¬ 
pected  value  of  inX,  is  the  arithmetic  mean,  which  is  defined  by 
the  equation  n 


where  z  =  £nX^ 


/nX4 


Tn  ‘  -■ 


1=1 


n 


n 

2  z 
1=1  ‘ 
n 


(5) 


Then  the  likelihood  of  T^  is 

n 


L(t  I  e)  =  -^1  e"  ^  (Tn  e) 

n  c^r 


(6) 


From  Equation  4,  if  T*  is  the  critical  value,  and  y(0)~to(0,  w 
then 


f(elTns  T*)  =  £ 


TT 


e  2w^  J 


n 


-oo 


-  2, 


<V®>‘ 


a-psr 


dTn 

(7) 


where 


K 


00 

=  ru_ 

j  wvFT 


.uaai  „ 


T* 


VS 


(Tn 

-ar  n 


-  e)4 


a  7  2  7r 


dTnde 


(8) 


To  evaluate  Equations  7  and  8,  the  following  approximation 
for  the  cumulative  normal  probability  function,  Reference  10, 
will  be  used: 


r  ^ dv - k 

u-oo  7  2  7T 


i+^vx  l-  e 


(9) 
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where 


Q(V)  = 


f-i  if  vs  o 
1  if  V>  0 


After  the  normalizing  transformations 


^  (T  -0) 

tn  =  - - 5 -  and  y 


e-3 

W 


are  made  and  the  cumulative  normal  approximation  (Equation  9)  is 
applied,  Equation  7  reduces  to 

f(9|TnsT*)  =  f  (y  lTngT*) 

-  k  ik  e  'y2/2  [i  -  e(B-y»)(i-e  -A(B-yw)2)1/2' 

(10  J 

where 


and  B  «  T* 


e 


Prom  the  defining  normalizing  property  of  K, 


2K  -  f  00  —  e  -y  /2  ["  1  +  Q(B-yw)  (l  -  e"A^B_yw^  f^2  dy 

2 

«  1  +  Q(B-yw)  (l  -e  -A(B-yw)2>)  /dy. 

To  evaluate  K,  the  series  expansion  is  used: 

(1  -  X)2  =  1  -  X/2  -  X2/8  -  ...  ~  CkXk  -  ...  ,  I X  I <  I 

00  k 

=  1-2  C.  X 
i=i  K 


1(1.5  -  k)C.  1 

where  C±  -  1/2,  Ck  =  | - g - —  ,  k  =  2,  3, . . . 

imating  the  term  ^1  -  g"A(B-yw)  ^ 


for  approx- 


After  squares  are  completed  and  the  cumulative  normal 
approximation  is  used,  the  final  result  for  the  posterior 
density  is  as  follows: 
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f(9lTsT*)  = 


1 


or  _2n  r  , 

(fl-fl )  |  .  f  7i o?  (T*-e )—  V2 


Fb^I/2  ~ 
l+Q(B/w)(  l-e_7T  /  _  2  c  /e  e 

k='l  k  k 


-kAB2E, 


2  2 
-  R,.\l/2 


kQ(Rk/ i-e'  "k' 

where  m  .  an  integer  large  enough  to  yield  a  good  approbation 
for  the  series  expansion  of  ^  Q  -A(3-yw )^l/2 

Ek  =  (2kAw2  +  l)'1 
Rk  =  B/w  -  2kARSkw 

7'5'6  £grlvatl°n  of  Oumulatlve  Posterior  iW'-'.ntlrn 


1  n 

Since 


]. 


Ptec^lT r'e\ Tn^r*)de , 


-  00 


from  Equation  10 


P[W9llTn!ir*1  "  §K  J1  /2  l-UfB-ywlCl-g-AfB-yw)2^2 

“  00  / 


By 

(12) 


i!te!  y,  ‘  ' (^'"5)/w-  10  avoW  havine  to  use  Laplaoe-Stleltjeo 

mutually aeXolSsive°oases  ya*Vw°andhy  s  |'V'ction<  ‘he  two 
separately.  y^sc/w  ana  y^>  B/w  are  -jonsidered 
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Case  1. 


B/w 

For  Case  1,  Q(B-yw)  =  1  over  the  range  of  integration  on  y 
and  Equation  12  can  be  rewritten  as 


P[0=se 


rV**] 


1_  f 

2K  J 


7'2 


l+^.e-A(B-y  «)2/2l 


dy  (13) 


The  result  of  using  the  series  expansion  for  the  radical 
and  completing  the  square  in  the  exponent,  as  was  done  for 
evaluating  K,  is 

F[es0,lT  tf*]  « 

-L  l  A 


1 

2k 


yi 


2/  e~y  /2  ^  -  s  c  ££e-kAB  Eic r  — —  e'^k 

4  vT*  k=1  4  ^ 


.^“(y-2kAEEkw): 


Employing  Equation  9  after  standardizing  the  noimal  densi¬ 
ties  resuits  in 

r  2  2 

P[e*91lTn*T*]  -  yif  2J 

-  z  C.VET  e_kAP  Ek(i-tQ(zvy'i-e'rfak  *k 

k=l  K  k  k  V 


i-  z  2  1/2 


(14) 


where 


-  y1  -  2kABEkw 

Case  2;  y^  B/w 

Per  Case  2,  the  following  equation  is  first  considered: 


00  v2 
-y  /2 


2  J/2. 


p[e>*llTnST*3  f  l4^(B-ywA-e'A(B'yw)  ^ 

A  J  „  Ar  L  \  ' 


dy 

(15) 
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Since  ,y1  >B/w  over  the  range  of  integration, Q(B-yw)  =  1. 

Employing  exactly  the  same  approach  as  for  Case  1  results 
m 


m 


P[8>9l'VT*]  =  7®? 


k=l 


ck^ve~kkJi2\ 


i-e 


-SET  zk 


1/2 

\ 

) 

(16) 


Then 


Pte^lT^T*]  =  1  -  P[0-61iTnsT*] 

Substituting  for  K  produces  the  following  final  results: 


yx  sB/w 


P[9s51lTnsT*] 


2r 


^•KKyppfyp]- 

l-Ki(B/w)i(B/w)  - 


(18) 


5X>  E/w 

- r-  m 

S  Ok[l-Q(Zk)F(Zk/VEJ)] 

P[«91ITn<T*]  =  1  -  i=i - - - 

£^l+3(B/«)F(E/w)|-  S  QkQ(Rk)P(Rk//^) 

(19) 


where 


T*  -  e 

1/2 

(l.5-k)Ck_1 

:  - g— - 

>  (2kAw2+l)‘1 


k>l 


-2  1/9 
=  ^i-e  *  r )  A 

ok/E^  e_kAB\ 

r-i  if  xso 

3  if  X  >0 
B/w  -  2kABEkw 

y1  -  2kAHE'kw 
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7 . 5 • 7  Maintainability- Demonstration-Test  Application 

The  results  of  Subsection  7-5 .6  can  now  be  used  to  find  the 
sample  size  n  arrd  critical  value  T*  such  that 

P [f  $tH  0=  Q  ]  =  1  -  a  (20) 

1  n  o J 


where 


'[e>81lTnsT»]  =  eb 


£  inx  £  7 , 

~  1-1  1  i=l  1 


2  2 

Since  z^  is  N(0,cr  ),  Tn  is  N(0,a  /n),  then  the  requirement  ex¬ 
pressed  by  Equation  20  can  be  rewritten  as 


/  Sn_  c-  (V9o>‘ 

-oo  V2no 


dT  =  1  -  a 
n 


where  $  ( 
tion. 


/T*  -0. 


-  1  -  a 


W-nrj 

)  denotes  the  cumulative  normal  probability  func- 


If  Za  equals  the  normal  deviate  defined  by  0(ZQ)  =  1  -  a, 
then,  from  Equation  22, 

T*  -0 

- 2.  =  z 

c  /Vn  a 


T*  =  eo  +  Za 


(23) 


The  second  test  requirement  (Equation  21)  is  equivalent  to  the 
equality 


i  -  pto^iTsT*]  =  i  -  eb 

where  P  [0:50^1  T^sT*]  16  obtained  from  Equation  18  or  19. 


(24) 
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Equations  23  and  2^  are  then  sufficient  for  determining  the 
n  and  T*  values  to  satisfy  the  c.  and  risks.  A  generalized 

computer  flow  chart  for  obtaining  n  and  T*  giver,  u,  (3,  ,  q  $  , , 

_  ■  i)  2  ’  D  O  X 

9,  T/,and  a  }is  presented  in  Figure  7. 

Standardized  sampling-plan  tables  have  been  developed  by 
using  this  flow-chart  procedure.  In  Equation  12,  it  is  seen 
that  P[0S0 ^1  T  sT*l  depends  on  only  three  constants: 

9,-6 
_  1 


B  =  T*  -  e  =  3o  ~  G  +  Zoa///?r 

W  W  W  V.’ 


(from  Equa¬ 
tion  23) 


For  a  given  a,  the  above  three  constants  determine  an  equivalent 
set  of  constants  that  can  be  used  as  indexing  parameters* 


X  = 

Y  = 

Z  = 


■0o 

n 

l  - 

01 

w 

0,. 

9 

1 

0 

w 


Sampling  plans  in  the  form  (n,  T*)  can  then  be  developed 
from  the  entires  a,  p,  z,  and  Y  to  yield  a  value  for  X. 

Then,  from  the  definition  of  X,  the  sample  size  is  found 
from  the  equation 


n 


(q-e0) 


and  then  given  n,  T*  i3  found  from  Equation  23.  Table  XLII  pre¬ 
sents  the  X  valueB  for  all  combinations  of  the  following  param¬ 
eters: 
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Read  Test-Criterion 

Parameters: 

e0’ 


Read  Prior-Distribution 

p 

Parameters,  F,  w  and 

2 

Constants  o  ,  M 


/  Does  \ 
?rior  Meet 
Beta 
Require - 
\  ment?  / 


n  =  u  +  1 


Compute  T*, 
Eq.  23 


^Sq.  18  *  <.Eq.  19  *  1-f 


Print  n,  ?* 


FIGURE  7 

PLOW  CHART  PCJR  COMPUTER  SCJUTICK  POR 
SAMPLE  SIZE,  n,  AND  CRITICAL  VALUE,  T* 
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a  =  0.05,  0.10,  0.20 

Pb  =  0.01,  0.05,  0.10,  0.20,  0.50 
Y  •■  -1,  -0.5,  o,  0.5,  1,  2 

Z  -  0.10,  0.25,  0.50,  0.75,  1.0,  1.5,  2.0,  2.5,  3.0,  4.0 

A  value  of  X-0  In  the  table  signifies  that  the  prior  dis 
tribution  already  meets  the  Bayesian  risk  requirement ,  thus 
obviating  the  need  for  testing. 


7*5.8  Example 

Assume  that  corrective-maintenance  time  for  an  avionic 
equipment  is  lognormaliy  distributed.  A  maintainability-demon¬ 
stration  test  is  to  be  performed  with  the  requirement  that  if 
the  mean  corrective-maintenance  time  p.  is  1/2  hour,  there  will 
be  a  95#  acceptance  probability.  Ihere  will  also  be  99# 
assurance  that  an  accepted  product  will  not  exhibit  a  mean  re¬ 
pair  time  greater  than  one  hour. 

2 

An  estimation  procedure  leads  t,o  an  estimate  of  O.75  for  a  . 
Prior  information  (e.g.3that  from  Table  XLIIl)  indicates  that  the 

normal  parameter  values  "e  =  -0.45  and  w2  =  0.30  can  be  used  for 
the  prior  density. 

The  inputs  resulting  from  the  above  are  as  follows: 


a  -  0.10,  ^  *  0.01,  jjq  —  1/2,  —  1.0 


Prom  the  equation  6 


in  p  -  o2/2 


; 


e0  -in  0.50  -  0.75/2  -  -1.068 
e±  -in  1.0  -  0.75/2  -  -0.375 

Thus  the  test  requirements  are 


P[Tn*r*le  -  -1.068]  -  0.95 

* 

P[0a-O.375}TnrfD«3  -  0.01 


( 
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where 


T  =  the  mean  of  the  natural  logarithm  of  n  observed 
n  corrective-maintenance  time 

The  inputs  needed  to  use  Table  XLII  are  as  follows: 


a  *s  0.05 


^  *=  0.01 


Y 


-0.45  +  0.375 


0.137 


„  el  “  e0  -0.375  +  1.068 
L  =  -  = - trgTiB - =1.265 

Por  conservatism,  the  next  higher  tabular  entries  of  Y  =  0  and 
Z  «  1.5  were  used,  leading  to  the  result  X  =  11. 275.  Then 


n  -  a2  -  .  UL25ll°J5)  ,  18 

(Sj-Sq)2  (0.693) 

The  critical  value  is  then 

T*  *=  0O  +  Zaa/^ 

-  -1.068  +  1.645  (0.866/4.242) 

-  -0.732 


Thus,  a  random  sample  of  18  corrective  maintenance  actions  are 
observed.  The  sample  mean 


is  ocaaputed .  If  TQg  -0.732,  the  equipment  is  accepted;  otherwise 
a  reject  decision  is  made. 


mmiwi  mmwmmmmm 
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TABLE  XLII 

X  VALUES  FOR  CALCULATl VO  SAMPLE  SIZE,  n 


n  = 

ac 

Y 

A 

y  =-i 

•  000 

ALPHA 

»  0.200 

z 

0.500 

0.200 

SETA 

0.100 

0.050 

0.010 

0.10 

0.25 

0.50 

0.75 

1.00 

1.50 
2.00 

2.50 
3.00 
4.00 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

C-0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.020 

0.060 

0.160 

0.255 

0.358 

0.515 

0.633 

0.706 

0.774 

1.367 

0.100 

0.343 

0.645 

0.911 

1.182 

1.651 

2.069 

2.466 

2.859 

3.630 

0.870 

1.747 

2.605 

3.182 

3.648 

4.430 

5.135 

5.807 

6.468 

7.750 

ALPHA  w  0.100 

Z 

0.500 

0.200 

BETA 

0.100 

0.050 

0.010 

0.10 

0.25 

0.50 

0.75 

1.00 

1.50 
2.00 

2.50 
3u00 
4.00 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.040 

0.140 

0.325 

0.491 

0.656 

0.944 

1.182 

1.356 

1.478 

0.190 

0.600 

1.124 

1.485 

1.802 

2.438 

2.957 

3.425 

3.868 

4.724 

1.410 

2.632 

3.786 

4.514 

5.079 

5.978 

6.752 

7.4T7 

8.181 

9.557 

\  ' 

ALPHA  . 

0.050 

z 

0.590 

0.200 

BETA 

0.100 

0.030 

**#•*#•* 

0.010 

0.10 

0.25 

0.50 

0.75 

1.00 

1.50 
2.00 

2.50 
3.00 
4.00 

0.0 

OcC 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.060 

0.230 

0.528 

o.tta 

1.917 

1.411 

1.748 

2.029 

2.249 

2.523 

0.312 

0.895 

1. 444 
2.169 
2.943 
3,234 
3.803 
4.421 

4.  915 
5.840 

2.903 

3.568 

4.974i 

5.846 

6.495 

7.507 

8*34$ 

M16 

9.850 

11.281 

203 
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TABLE  XLII  (continued) 

Y  =*-0.500 


ALPHA  * 

0.200 

z 

BETA 

0.500 

0.200 

0.100 

0.050 

0.010 

0.10 

0.0 

0.020 

0.090 

0.290 

1.449 

0.25 

0.0 

0.060 

0.324 

0.739 

2.566 

0.50 

0.0 

0*170 

0.655 

1.403 

3.623 

0.75 

0.0 

0.277 

0.971 

1.911 

4.355 

1.00 

0.0 

0.369 

1.29C 

2.348 

4.956 

1.50 

0.0 

0.566 

1.853 

3*118 

5.994 

2.00 

0.0 

0.697 

2.370 

3.821 

6.945 

2.50 

0.0 

0.896 

2.868 

4.492 

7.857 

3.00 

0.0 

1.234 

3.356 

5.145 

8.738 

4.00 

0.0 

1.789 

4.275 

6.358 

10.401 

ALPHA  « 

0.100 

l 

BETA 

0.500 

0.200 

0.100 

0.050 

0.010 

0.10 

0.0 

0.030 

0.170 

0.489 

2.210 

0.25 

0.0 

0.120 

0.545 

1.230 

3.720 

0.50 

0.0 

0.315 

1.093 

2.107 

5.043 

0.75 

0.0 

0.494 

1.499 

2.768 

5.927 

1.00 

0.0 

0.676 

1.891 

3.340 

6.625 

1.50 

o.c 

0.997 

2.631 

4.254 

7.782 

2.00 

0.0 

1.258 

3.256 

5.058 

8.806 

2.50 

0.0 

1.443 

3.835 

5.806 

9.773 

3.00 

0.0 

1.609 

4.388 

6.523 

10.715 

4.00 

0.0 

2.266 

5.450 

7.891 

12.510 

ALPHA  « 

0.050 

Z 

•»••»««« 

>•••«••••• 

SETA 

0.500 

0.200 

0.100 

0.050 

0.010 

0.1 

0.0 

0.QS0 

0*250 

0.724 

2.985 

0.1 

0.0 

0*200 

0.895 

1*751 

4.882 

Q«5Q 

0*0 

0*487 

1*564 

2.825 

6*496 

0*  ?5 

0*0 

0*759 

2*139 

3*690 

7.465 

ioOO 

0*0 

1*008 

2*571 

4*353 

6.245 

t*$0 

0*0 

1*450 

3*432 

5*409 

9*510 

2.00 

0.0 

1*826 

4.175 

6.299 

10.600 

2*  50 

0*0 

2*131 

4*828 

7*112 

11.615 

3*09 

0*0 

2.360 

5*437 

7*882 

12*595 

4*09 

0*0 

2.664 

6*594 

4*350 

14*482 
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TABLE  XLII  (continued) 


_  .  ..ALPHA  »  0.2C0 

i _  •»*#»*••»*#»»##»» 


0.500 

0.200 

0.10 

0.0 

0.060 

0.25 

Q.O 

0.230 

C.50 

0.0 

0.547 

0.75 

0.0 

0.842 

1.00  ' 

‘0.0 

1.169 

1.50 

o.c 

1.769 

2.00 

0.0 

2.337 

2.50 

0.0 

?  .  6  86 

3.00 

0.0 

3.413 

4.00 

0.0 

4.374 

ALPHA  - 

0.100 

2 


0.500 

0.200 

0.10 

0.0 

o.nc 

0.25 

0.0 

0.412 

0.50 

0.0 

0.869 

0.75 

0.0 

1.307 

1.00 

O.C 

1.669 

1.50 

0.0 

2.447 

2.00 

0.0 

3.138 

2.50 

0.0 

3.779 

3.00 

0.0 

4.396 

4.00 

0.0 

5.545 

ALPHA  «• 

0.050 

l 

«••««•* 

0.500 

0.200 

0.10 

0.0 

0.160 

0.25 

0.0 

0.569 

0.50 

C.O 

1*248 

0*79 

0.0 

1.820 

1*00 

0.0 

2.289 

1.50 

0*0 

3.128 

2.00 

0.0 

3.956 

2*50 

0.0 

4.667 

3.00 

0,0 

5.372 

4*0r 

0.0 

6.662 

203 


BETA 

0.10C 

0.050 

0.010 

0.210 

0.507 

1.989 

0.634 

1.242 

3.289 

1.251 

2.138 

4.518 

1.795 

2.825 

5.399 

2.275 

3.422 

6.138 

3.149 

4.481 

7.438 

3.956 

5.457 

8.636 

4.726 

6.379 

9.774 

5*453 

7.243 

10.857 

6.778 

8.833 

12.869 

BETA 

0.100 

0.050 

0.010 

0.384 

0.861 

2.958 

1.014 

1.880 

4.636 

1.849 

3.074 

6.129 

2.573 

3.929 

7*148 

3.174 

4.640 

7;977 

4.207 

5.854 

9.401 

5.131 

6.936 

10.680 

5.996 

7.950 

11.896 

6.823 

8.914 

13.058 

6.335 

10.678 

15.216 

BETA  «<»••••••#•&«••••• 


o.l  00 

0.050 

0.010 

0.533 

1.222 

3*940 

1.440 

2.559 

5*965 

2.503 

4.028 

7*694 

3.366 

5*041 

OrSSS 

4.091 

5.856 

9.749 

5*276 

?.?06 

£.1.275 

6.  300 

0.380 

12*625 

7*244 

9.467 

13*895 

8*143 

10.499 

15*114 

9.806 

12.411 

17*396 

TABLE  XLII  ( continued) 


Y  *  0*500 


z 

ALPHA  * 

0.200 

BETA 

0.500 

0.200 

0.100 

0.050 

0.010 

0.10 

0.020 

0.130 

0.386 

0.752 

2.480 

0.25 

0.060 

0.461 

0.979 

1.726 

3.930 

0.50 

0.180 

0.989 

1.856 

2.826 

5.326 

0.75 

0.317 

1.520 

2.573 

3.665 

6.352 

1.00 

0.450 

2.009 

3.217 

4.436 

7.230 

1.50 

0.698 

2.926 

4.392 

5.782 

8.804 

2.00 

1.077 

3.782 

5.467 

7.008 

10.242 

2.50 

1.468 

4.584 

6.461 

8.142 

11.586 

3.00 

1.821 

5.325 

7.381 

9.188 

12.639 

4.00 

2.430 

6.697 

9.091 

11.146 

15.215 

ALPHA 

=  0. 

100 

Z 

BETA 

0.500 

0.200 

0.100 

0.050 

0.010 

0.10 

0.030 

0.220 

0.612 

1.231 

3.605 

0.25 

0.110 

0.725 

1.485 

2.536 

5.420 

0.50 

0.296 

1.461 

2.645 

3.932 

7.072 

0.75 

0.500 

2.136 

3.537 

4.957 

8.234 

1.00 

0.715 

2.755 

4.305 

5.825 

9.209 

1.50 

1.116 

3.850 

5.652 

7.334 

10.909 

2.00 

1.442 

4.844 

6.865 

8.689 

12.449 

2.50 

1.783 

5.772 

7.986 

9.942 

13.892 

3.00 

2.344 

6.638 

9.026 

11.103 

15.239 

4.00 

3.219 

8.189 

10.903 

13.214 

17.742 

ALPHA 

±  0. 

050 

Z 

n»**« 

>»»•••«* 

BETA 

0.5C0 

0.200 

0.100 

0.050 

0.010 

0.10 

0.040 

0.343 

0.870 

1.734 

4.728 

0.25 

0.160 

1.017 

2.067 

3.362 

6.880 

0.50 

0.432 

2.000 

3.448 

5.043 

8.755 

0*75 

0.716 

2.746 

4.515 

6.219 

10.041 

i.oc 

0.990 

3.513 

5.396 

7.190 

11.096 

1.50 

1.533 

4.701 

6.894 

8.838 

12.901 

2.00 

2. 005 

5.093 

0.217 

10.294 

14.518 

2.50 

2.303 

6o  922 

9.440 

11.641 

16.034 

3.00 

2.692 

7.005 

10.577 

12.094 

17.461 

4«rQC 

3.904 

9.613 

12.615 

13.153 

20.083 

Y  -1.000 


TABLE  XLli- (continued) 


-  -  ALPH*  »  0.  200 


L 


0.10 

0.25 

0.50 

0.7S 

1.00 

1.50 

2.00 

2.50 
3.00 
4.00 


l 


0.10 

0.25 

0.50 

0.75 

1.00 

1.50 

2.00 

2.50 
3.00 
4.00 


Z 


0.10 

0.25 

0.50 

0.75 

1.00 

1.50 

2.00 

2.50 
3.00 
4.00 


-E.JLA 

o.ioo 


0.500 

0.040 

0.16C 

0.439 

0.719 

1.063 

1.745 

2.390 

2.986 

3.542 

4.605 


0.200 

0.220 

0.696 

1.479 

2.190 

2.847 

4.060 

5.160 

6.158 

7.037 


0.533 

1.363 

2.443 

3.342 

4.140 

5.603 

6.903 

8.076 

9.171 


8.875  ..  11.282 


ALPHA  *  0.100 


0.50C 

►  B  JE  T  A 

0.200 

0.100 

0.060 

0.386 

0.872 

0.250 

1.078 

1.982 

0.661 

2.080 

3.381 

1.070 

2.974 

4.455 

1.449 

3.762 

5.391 

2.238 

5.179 

7.040 

3.101 

6.450 

8.506 

3.845 

7.601 

9.628 

4.525 

8.649 

11.034 

5.742 

10.582 

13.282 

ALPHA  « 

0.050 

BETA 

0.500 

0.200 

0.100 

0.090 

0.523 

1.223 

0.370 

1.461 

2.640 

0.899 

2.689 

4.330 

1.430 

3.771 

5.568 

1.928 

4.687 

6.617 

2.783 

6.275 

8.430 

3.786 

7.667 

10.028 

4.664 

8.965 

11.471 

5.462 

10.126 

12.783 

6.860 

12.205 

15.161 

0.050 

0.010 

1.043 

2.931 

2.193 

4.511 

3.489 

6.080 

4.516 

7.259 

5.427 

8.290 

7.044 

10.130 

8.483 

11.779 

9.786 

13.287 

11.002 

14.707 

13.363 

17.483 

0.050. 

0.010 

1.600 

4.177 

3.141 

6.122 

4,733 

7.938 

5.932 

9.255 

6.958 

10.383 

8.756 

12.365 

10.354 

14.143 

11.794 

15.766 

13.116 

17.275 

15.598 

20.153 

•••»•<»«+ 

0.050 

0.010 

2.224 

5.419 

4.102 

7.684 

5.967 

9.719 

7.315 

11.159 

6.445 

12.365 

10.394 

14. 460 

12, 112 

16.336 

13.666 

18.055 

15.084 

19.649 

17.682 

22.613 

1JT219X 
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TABLE  XLII*  (concluded) 


Y  «  2.000 


ALPHA  *  0.200 


l 

BETA 

0.500 

0.200 

0.100 

0.050 

0.010 

0.10 

0.110 

0.478 

0.976 

1.692 

3.844 

0.25 

0.456 

1.345 

2.211 

3.195 

5.698 

C.50 

1.129 

2.576 

3.722 

4.898 

7.622 

0.75 

1.809 

3.635 

4.957 

6.245 

9.U4 

1,00 

2.449 

4.578 

6.036 

7.419 

10.407 

1.50 

3.608 

6.216 

7.890 

9.421 

12.638 

2.00 

4.654 

7.651 

9.495 

11.157 

14.580 

2.50 

5.657 

9.005 

11.009 

12.788 

16.410 

3.00 

6.650 

10.341 

12.501 

14.40C 

18.222 

4.00 

8.604 

12,964 

15.452 

17.603 

21.865 

ALPHA  - 

0.100 

2 

BETA 

0.500 

0.200 

0.100 

0.050 

0.010 

0.10 

0.170 

0.735 

1.472 

2.483 

5.315 

0.25 

0.660 

1.901 

3.104 

4.391 

7.513 

0.50 

1.518 

3.461 

4.929 

6.393 

9.685 

0.75 

2.397 

4.723 

6.361 

7.925 

11.325 

l.CC 

3.198 

5.830 

?.594 

9.236 

12.731 

1.50 

4.608 

7.724 

9.689 

11.468 

15.155 

2.00 

5.830 

9.336 

11.466 

13.366 

17.236 

2.50 

6.946 

10.801 

13.085 

15,095 

19.153 

3.00 

8.023 

12.215 

14.649 

16.771 

21.017 

4.00 

10.139 

14.995 

17.736 

20.097 

24,747 

ALPHA  * 

0.050 

L 

•#*••*»< 

BETA 

0.500 

0.200 

0.109 

0.050 

0.010 

0.10 

0.249 

1.019 

2.020 

3.295 

6.757 

9.25 

0.878 

2.489 

4.004 

5.530 

9.260 

0,50 

1.960 

4.354 

6.121 

7.846 

11.643 

0.75 

2.959 

9.797 

7.719 

9.532 

13.404 

1.00 

3.929 

7w©40 

9.061 

10.962 

14.905  ' 

1.50 

5.572 

9.155 

11,382 

13.362 

17.477 

2.00 

6.945 

10.929 

13.312 

15.421 

19.664 

2.50 

8.199 

12.502 

15,030 

17.243 

21.678 

3.00 

9.393 

13.986 

16.658 

10.977 

23.591 

4*00 

11.589 

14.880 

19.850 

22.399 

27.400 

IJT2I9I 
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7-5*9  Prior- Distribution  Analysis 

_•  p 

The  i-rior-dlstributlon  input  of  the  parameters  eT  and  w  is 
a  controlling  factor  In  the  teat.  In  fact,  as  shown  by  the  tab¬ 
ular  values  of  Table  XLIII,  X  *  0  if  the  prior  distribution  is 
good.  Then,  theoretically,  no  testing  is  necessary,  although  ir 
practice  this  decision  would  not  normally  be  made. 

Prior- distribution  development  has  always  been  a  trouble¬ 
some  area  in  Bayesian  statistics.  However,  the  evidence  that 
Bayesian  methods  are  being  used  more  than  ever  (e.  g.,  in  reli¬ 
ability  demonstration)  will  provide  impetus  to  developing  appro¬ 
priate  procedures  and  data.  In  addition,  the  current  emphaslB 
on  establishing  centralized  Government  and  industry  reliability 
and  maintainability  data  banks  will  provide  a  good  source  of 
historical  maintainability  data. 

At  the  moment,  the  following  are  possible  sources  of  in¬ 
forms  tion  for  us  »  in  developing  prior  distributions: 

.  Maintainability  predictions  and  assessments 

.  Previous  demonstration  tests 

.  Observed  maintainability  of  similar  equipments 

Design  and  development  tests 

v. 

2 

.  ^objective  evaluations 

These  sources  are  discussed  briefly  in  the  following  paragraphs. 

Maintainability  predictions  and  assessments  can  provide  use 
ful  data  through  (1)  analysis  of  past  relationships  between  pre¬ 
dicted  and  observed  maintainability,  (2)  confidence-interval 
estimates  such  as  those  provided  through  prediction-by-function 
approaches,  and  (3)  use  of  several  acceptable  sets  of  prediction 
input  data. 

Procedures  are  available  for  using  the  results  of  previous 
demonstration  tests  as  prior  information  for  subsequent  tests. 
This  is  a  natural  application  of.  Bayesian  statistics. 

Products  similar  to  those  Under  test  are  an  Important  data 
source.  Their  observed  maintainability  characteristics  are 
valuable  data,  but  differences  in  design,  part  reliability,  de- 

•  >  sign  tea turlty,  environment,  etc.,  must  be  accov~*ed  for. 

*■  As  the  equipment  progresses  through  development,  various 

engineering  tests  are  performed  on  parti,  assemblies,  and  eam- 

*  ponents.  While  these  teats  may  not  be  designed  to  provide  es¬ 
timates  of  maintainability  parameters,  such  estimates  can  be 


obtained  if  appropriate  recording  procedures  are  established. 

For  example,  all  maintenance  times  on  a  design- conformance  test 
should  be  recorded  to  establish  benchmarks.  This  information  can 
then  be  U3ed,  for  example,  to,  adjust  maintainability  predictions 
based  on  paper  designs . 

Subjective  evaluation^  are  quite  controversial,  but  at  this 
time  it  is  unlikely  that  sufficient  directly  applicable  data  are 
available  for  developing  prior  distributions;  engineering  judg¬ 
ment  will  be  required  to  adjust  existing  data  to  the  problem 
and  to  use  qualitative  appraisals  as  necessary.  In  any  case,  it 
is  important  that  the  producer  and  consumer  agree  on  the  appli¬ 
cability  and  realism  of  subjective  e/aluations  used  for  prior- 
distribution  analysis.  Research  is  being  conducted  on  the 
quantification  of  personal  judgments  for  Bayesian  applications, 
and  specific  approaches  have  been  developed  (see  Reference  11 
for  an  example ; . 

—  2 

Three  very  simple  methods  for  estimating  6  and  w  are  out¬ 
lined  below. 

7.5 - 9-1  Method  1  -  Use  of  Historical  ;Data 

The  maintenance-time  data  that  were  collected  for  21  equip¬ 
ments  during  this  study  have  been  analyzed  to  obtain  values  for 

.i  p 

5yand  w  ,  the  mean  and  variance,  respectively,  for  the  normal 
prior  density  required  for  use  in  the  BayeBian  teBt.  These 
values  are  presented  in  Table  XLIII  for  eight  categories  repre¬ 
senting  various  combinations  of  environment,  maintenance  index, 
and  inclusion  or  exclusion  of  no- trouble- found  actions. 

In  the  absence  of  more  pertinent  data,  these  values  can  be 
used  as  prior  information  or  can  serve  as  initial  valueB  to  be 
modified  by  predictions,  subsequent  development  tests,  and  sub¬ 
jective-type  analyses. 

7 .5 - 9.2  Method  2  -  Use  of  Predictions 


Maintainability-prediction  procedures  such  as’those  pre¬ 
sented  in  MIL-OTD-472  and  in  Volume  1  of  this  report  constitute 
a  means  for  obtaining  a  normal  prior  if  "pessimistic"  and  "op¬ 
timistic"  prediction  inputs  can  be  reasonably  calculated  so  that 
pessimistic  and  optimistic  predictions  will  be  obtained  for  the 
mean  p.  or  median,  M. 

2 

In  Is  recalled  thato  is  assumed  to  be  known  either 
through  use  of  the  tabular  values  given  m  Table  XXVI  or  through 
the  prediction  procedure  of  Subsection  7.3*5. 
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TABLE  XLIII 


/ 


OBSERVED  MEAN  AND  VARIANCE  OF  0  FOR 
NORMAL  PRIOR- DISTRIBUTION  ANALYSIS* 


System  Type 

No -Trouble- 
Found  Actions 
Included 

Nq- Trouble- 
Found  Actions 
Excluded 

Mean, 

0 

Variance, 

2 

w 

Mean, 

0 

Variance, 

2 

w 

Avionic  Systems 

Corrective-Maintenance 

Time 

-0.451 

0.299 

-0.297 

0.287 

Corrective -Maintenance 

Man- 

Hours 

0.007 

0,550 

0.165 

0.540 

Ground  Systems 

Corrective-Maintenance 

Time 

-0.442 

0.214 

-0.545 

0.190 

Corrective-Maintenance 

Man 

Hours 

0.058 

0.226 

0.129 

0.167 

*A11  0  figures  shown  are  based  on  times  recorded  in  hours.  To 
convert  to  minutes,  add  4.094  to  the  tabular  value.  No  con¬ 
version  is  necessary  for  the  variance  values. 


Given  two  predictions  of  the  mean  —  say,  M-j/  the  lower  or 
pessimistic  value,  and  p.y,  the  upper  or  optimistic  value  —  there 
are  two  equivalent  predictions  of 


=  in  jir,  -  a2/? 

G-rj  =  fn  Uy  -  o  /2 


If  median  values  are  predicted  — 


say  and  My  — 


then 


7*0 


I 


.  -Ha 


'  ■  l 
:  >• 


*-.s-w v ■■  m 

m&m 
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where  Z^g  la  the  nonnal  deviate  corresponding  to  the  (l-p/2) 
percentile . 

Another  procedure  is  to  use  a  predicted  value  for  \i,  sa 
and  100x(l-p)#  confidence  limits  for  a2  from  the  predictio 
procedure  of  Subsection  7.3.5.  If  a\  and  CT2y  represent  the 
100x(l-p)^  confidence  limits,  then 


and,  as  above 


. enviw 


w 


( 


where  Zp/g  in  this  case  repre sente  the  nomal  deviate  correspond¬ 
ing  to  a  lOOx(l-p)#  confidence  1  «vel  associated  with  the  c-2T  and 
o  o 

a  n  values.  The  value  of  a  to  uae  In  the  test  should  he  the 

u  ..2 

point  estimate  d  unless  there  is  reason  for  using  a  more  con¬ 
servative  value. 

7 -5 -9-3  Method  3  -  Subjective  Methods 

Instead  of  predictions,  subjective  evaluations  say  be  used 
_  o 

for  obtaining  9  and  w  .  For  example,  suppose  the  following  is 
believed  to  be  reasonable: 

(a)  There  Is  &  50-50  chance  that  the  mean  corrective-, 
maintenance  time  is  less  than  1  hour. 

(b)  There  is  only  a  10-percent  chance  that  the  mean 
is  over  1.5  hours. 

(c)  There  is  only  a  1-percent  chance  that  the  mean  IS 
.less  than  1/2-hour. 

2 

From  the  relationship  p,  -  e®+  a  ^ ,  tne  following  are 
equivalent  statements: 

(a)  P[e  0+  a  /2  *0.53  *  0.01 

(b)  P[e  9‘kj2-/2  *  l.o]  *  0.50 

(c)  He  /2  *  1.5]  *  0.90 


2 

A&suae  &  value  of  0.8  for  a  . 

Taking  logarithms  and  substituting  numerical  valuer  lecus 
to  the  following: 

(a)  F[0  g  -  1  093]  *  0.01 

(b)  F[e  si  -  0.4]  «  0.50 

(c)  i{  9  *0.00Q  «  0.90 

Frcai  the  normal  prior  assumption  for  W,  the  (b)  relation¬ 
ship  establishes  that  -  -0.4.  Two  possible  w2  values  are  as 
follows : 
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Prom  (a)  and  (b) 


.tAj  .9S3  —I _ LO  aft )  „  2  _  2  t-  _g  3  3 

40.99  0.50 


w 


or 


w*  =  0.088 


From  (b)  and  (c) 


9^S19§.  ,C~Q«  0  «  2-  -  Z  <=  1  28 

*0.10  *0.50  x*^° 


w 


or 


w  «  0.101 


P 

Averaging  these  two  values  yields  an  estimate  of  w  =  0.095. 

7. 5*9-^  Comment 

'  It  is  emphasized  that  the  three  methods  described  above 
are  quite  simplified  and  that,  in  fact,  a  combination  of  all 
three  may  well  be  used  in  conjunction  with  maintenance  data  that 
may  be  available  on  similar  equipment. 

If  a  Bayesian  test  of  thi  type  described  is  to  be  performed, 
management  must  ensure  that  necessary  tests,  data  collection, 
and  data  analyses  take  place  charing  the  development  program  for 
use  in  establishing  a  prior  distribution  or  modifying  one 
developed  early  in  the  program. 
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SECTION  VIII 

test  akd  mtimmsMim 


GENERAL 


Various  administrative  and  procedure!  aspects  of  the  main¬ 
tainability-demonstration  program  are  discussed  in  this  section. 
Many  of  the  guidelines  and  reccaaswsdatione  are  a  result  of  a 
study  of  previous  demonstration-test  plans,  result©,  and  cri¬ 
tiques.13 

r  T  r  '  •*'* 

8.2  TEST  SCHEDULING 

o*  ■'  t  r 

»  ’•  <.«  "r  *  ' 

Table  XL IV,  which  is  abstracted  from  Attachment  1  of  Air 
Force  Regulation  80-14,  Test  and  Evaluation  of  Systems,  Sub¬ 
systems,  and  Equipment  (R&D),  summarizes  the  three  raa^or  Air 
Force  R&D  test  categories. 

Ideally,  a  maintainability  demonstration  should  be  sched¬ 
uled  for  each  test  category.  Category  1  tests  sfc  the  eSHijr&SEt 
level  can  provide  information  for  improving  the  maslnt alnabllity 
design  before  large-scale  production  build-up.  Further  design 
changes,  integration  pr  bleras,  and  maintenance  policies  and 
procedures  can  be  evaluaced  during  a  Category  II  test.  A  cat¬ 
egory  III  test  will  permit  as  realistic  a  measure  of  operatic nni  - 
system  maintainability  as  possible  for  final  verification. 
Information  is  also  provided  in  Category  IH  tests  for  evalaar* 
ting  the  adequacy  of  the  maintainability  support  program  in 
terms  of  training,  technical  manuals,  failure -reporting  proce¬ 
dures,  etc. 

By  the  nature  and  timing  of  the  tests,  a  Category  I  test 
will  probably  be  based  on  a  rauit-inuucement  ssDpling  procedure. 
Either  fault  inducement  or  natural  failures,  or  both,  are 
applicable  for  a  Category  II  test,  while  natural  failures  should 
be  the  primary  sampling  approach  for  a  Category  XIX  test. 

In  scheduling  the  msdLntainability-demonstraticn  test, 
consideration  must  be  given  to  other  test  retirements  («.g., 
reliability  demonstration)  end  the  usually  limited  number  of 
equipments  or  systems  available  for  testing  in  the  early  stages. 

The  tests  must  be  conducted  early  enough  so  that  sample-size 
requirements  can  be  met  and  time  is-  availwis  fp?  instituting, 
necessary  design  or  procedural  changes  as  a  result  of  the  test . 

On  the  other  hand,  the  tests  should  be  scheduled  so  that  neces¬ 
sary  training  ho©  taken  place,  doc$s®es»tsfcion  is  complete,  and 
information  for  establishing  fault  inducement  and  other  proce¬ 
dures  le  available.  tJnfortunately,  these  two  raqalrsm^ts  nwsy 
conflict  somewhat  and  a  compromise  schedule  may  have  to  be  adopted « 

13  A  particularly  valuable  reference  in  this  regard  Is  thy  report, 
naiat ainability-bcmcrnstrct ion  Result©  for  12  Electronic  £srrS®90J, 
rTTriEgS y7tr«pa5#rCTeFT5oESr32^^ 

WDL  Division,  December  1965. 
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A29TEBSITI0N  TBSnSG  -  APR  80-1$ 


Category  I  Testa 


Category  XU  Testa 


Category  II  Teste 


Subsystem*  Develop 
meat  Test  And 
1/eluation 


System  Operational  Test 
And  Evaluation 


Teat  and  evaluation  of 
operational  systems  by- 
operating  command. 

These  tests  Include  all 
components,  support 
items,  personnel  skills 
technical  data,  etc., 
and  will  be  performed 
under  as  near  opera¬ 
tional  conditions  as 
practicable.  Cat.  XU 
testing  will  be  con¬ 
ducted,  using  a  config¬ 
uration  as  Jointly 
agreed  by  the  operating 
command  and  AFSC/AFLO. 
Cat.  Ill  tests  will  be 
conducted  per  &/ specif¬ 
ic  test  plan.  Gat,  m 
testing  terminates  when 
preplanned  objectives 
of  acquisition  plan 
have  been  fulfilled. 


Development  test  and 
evaluation  spanning 
the  integration  of 
subsystems  into  a 
complete  system  in 
as  near  an  opera¬ 
tional  configuration 
as  practicable  under 
control  of  APSC. 
Suitable  instrumen¬ 
tation  will  be  em¬ 
ployed  to  determine 
the  functional  capa¬ 
bility  of  subsystems. 
Cat.  II  is  an  Air 
Force  effort  with 
contractor  partici¬ 
pation,  under  Air 
Force  direction  and 
coutrol,  and  with 
active  operating  and 
supporting  cccanand 
participation.  Ac¬ 
tual  test  operation 
and  maintenance 
should  b©  performed 
by  military  person¬ 
nel  who  have!  re¬ 
ceived  formal  ejr stem 


Development  test  and 
evaluation  of  the  in¬ 
dividual  components, 
subsystems  ami,  in 
certain,  cases,  the 
complete  system  un¬ 
der  control  of  AF3C. 
In  addition  to  quali¬ 
fication,  the  testing 
provides  for  rede¬ 
sign,  refinement,  and 
reeva^uation  as  ne¬ 
cessary  including  the 
practicality  of  using 
current  standard  and 
corsaorclai  items. 
These  tests  are  con¬ 
ducted  predominantly 
by  the  contractor, 
but  with  the  Air 
Force  active  parti¬ 
cipation,  evaluation, 
and  control. 


Control '  Systems  Com- 
i  a&nd 

[Participants: 

!  Contractor.  Ifeing 
(kc&tcndw.  Support 
CvmmA* 


Control:  Using  Commands 
Participants: 

AFSC,  Support  Com¬ 
mands,  Contractor 
When  Required 


Control:  ^sterns  Cor 
maud' 

Participants: 
.Contractor,  Using 
Cwaxandb,  Support 


8.3  PRIOR-INFORMATION  REQUIREMENTS 


The  use  of  prior  information  in  the  design,  corduct,  and 
evaluation  of  a  maintainability  demonstration  haa  been  dicouased 
throughout  this  report  *  Several  cf  the  more  important  applica¬ 
tions  are  summarised  in  Taole  XLVr 

It  should  be  a  continuing  effort  of  the  maJ^tainability- 
demonst  nation  management  group  to  ensure  that  all  available 
pertinent  information  is  properly  collected,  recorded,  end 
analyzed.  Specific  efforts  to  obtain  necessary  prior  informa¬ 
tion  should  be  planned  and  instituted  as  necessary  (e.g., 
special  tests  or  collection  of  data  on  similar  systems). 

8.4  MAINTENANCE-PERSONNEL  SELECTION 


To  the  extent  practicable,  the  personnel  involved  in  the 
maintainability  demonstration  should  tie  representative  of  thbm 
expected  during  normal  operation.  The  best  choice  would  be 
those  who  will  actually  be  assigned  to  the  equipment  for  then 
the  specific  training  and  experience  received  during  the  cktfson- 
atration  program  will  be  of  value  in  the  future*  Achieving* 
representativeness  involves  evaluation  of  skill  level,  education, 
general  maintenance  training  and  experience,  find  training 
experience  that  are  specific  to  the  equipment  in  question.  I’hese 
evaluations  should  apply  to  both  technicians  and  supervisory 
personnel . 

It  Is  most  desirable  that  the  selections  be  made  from  Air 
Force  personnel,  and  this  should  be  stated  in  the  contract  unless 
circumstances  prohibit  such  a  clause.  If  this  is  made  a  contrac¬ 
tual  requirement,  the  necessary  planning  for  selection,  training, 
and  indoctrination  can  be  completed  early  enough  so  that  even  a 
Category  I  test  can  be  performed  with  Air  Force  personnel.  The 
contractor  will  normally  require  that  he  be  allowed  approval  of. 
selected  personnel;  therefore,  biographical  information  should 
be  made  available  to  him. 


Another  consideration  concerns  the  number  of  personnel  to 
make  up  the  maintenance- team  pool*  The  greater  the  number,  fchn 
better-  fcne  chances  for  obtaining,  a  representative  stapling  of 
maintenance  times  when  technicians  are  assigned  to  tasks  i-sa- 
doaiy.  In  addition  to  cost  factors  there  is  a  limit  to  the 
number  of  technicians  that  should  be  available.  If  there  are 
too  many  technicians,  with  t\&  limited  sample  else  of  the  test, 
each  technician  will  perform  only  several  tasks  and  thus  the 
learning  that  wosild  be  acquired  in  operational  use  tfi  held  to 
a  minimum. 


It  is  recommended  in  the  Philec  Report**  that  a  maintenance 
tear  perform  no  more  than  six  tests  and  that  team  personnel  net 


fi  t  * 


'ABLE  XLV 


OF  PRIOR -INFORMATIO N  APPLICATIONS  AND  DATA  SOURCES 


Application 

Type  of  Information 
Required 

Sources 

Maintainability-Index 

Selection 

Overall  system  require¬ 
ments 

Concept  and  Defini¬ 
tion  Phase  studies 
Technical  develop¬ 
ment  and  program 
package  plans 

Maintainability - 
Demonstration  Re¬ 
quirements 

Data  for  determining 
re&w.^atic  and  consis¬ 
tent  index  values 

Maintainability  pre¬ 
dictions 

Historical  data 
Allocation  studies 

MaintainAM.13  ty- 
Bemonstration  Risk 
Yaluee 

Test  and  operational 
cost  data 

Historical  coat  data 
Cost  Predictions 
Logistic  analyses 

Selection  of  Statis¬ 
tical  Test 

. . . 

Maintainability -diotri- 
bution  data  and  avail¬ 
ability  of  test  inputs 

Historical  data 
Engineering  analyses 
Maintainability  pre¬ 
dictions  and 
analyses 

Sfanple  Size  and 
Decision  Criterion 

Expected  values  of  main- 
tainabillty -related 
parameters 
f  2  \ 

i  e.g. .  ojnjjfl) 

\  / 

Maint inability  pre¬ 
dictions  and 
analyses 

Historical  data 

Task-Sampling  Scheme 

, 

Maintenance -task  identi¬ 
fication  and  relative 
frequency  of  occurrence 

Engineering  analyses 
Maintainability  pre¬ 
dictions  and  analyses 
Historical  data 

Conduct  of  Test 

:-i .  1* . 

Information  for  achiev¬ 
ing  maximum  possible 
realism 

1 

Operational  plane 
Training -require - 
ment  studies 
Environmental 
studies 

be  mixed.  Tho  limitation  of  six  teats  is,  perhaps,  somewhat 
severe.  If  a  maintenance  team  consists  of  two  men,  and  50  tasks 
are  to  be  performed  (e.g„,  per  Test  Method  2,  MIL -STD- 4?!),  there 
would  have  to  be  at  least  nine  teams,  or  a  total  of  18  mainten¬ 
ance  personnel. 

There  is  no  single  correct  solution.  Consideration  must 
be  given  to  the  training  and  equipment-familiarization  program, 
frequency  of  expected  maintenance,  planned  manning  levels,  and 
the  learning  effects  of  time  compression.  Prom  past  demonstra¬ 
tions  involving  approximately  50  tasks  each,  four  or  five  teams 
appear  to  be  a  reasonable  number. 

8.5  DEMONSTRATION-REVIEW  TEAM 

The  team  designated  as  responsible  for  the  conduct  cf  the 
test,  and  for  observation  and  interpretation  of  test  results, 
must  be  carefully  selected.  Such  a  team  would  normally  consist 
cf  representatives  from  Air  Force  Contracts  Administration,  the 
equipment  contractor,  the  procuring  activity,  and  the  contractor. 
Representatives  of  the  using  command,  AFLC,  and  centralized  Air 
Force  maintainablxity-  or  effectiveness-assurance  offices  may 
also  be  on  the  team. 

An  Air  Force  representative  should  be  selected  as  the  re¬ 
view-team  director;  he  should  be  made  responsible  for  test  prep¬ 
aration,  overall  direction  of  the  test,  coordination  of  the 
review-team  activities,  and  preparation  and  submission  of  a  report 
of  the  demonstration  results. 


The  review  team  is  responsible  for  the  following: 

•  Observing  all  aspects  associated  with  the  maintenance- 
task.  occurrences  and  determining  whether  they  are  valid 
for  demonstration  purposes 

•  Recording  maintenance-tusk-time  data  and  other  informa¬ 
tion  pertinent  to  the  decision  criterion 

•  Making  decisions  to  handle  unexpected  circumstances  that 
may  arine 

•  Obtaining  and  recording  data  applicable  to  satisfying 
any  secondary  objectives 

•  Evaluating  the  results  of  the  demonstration,  preparing 
the  final  report,  and  recommending  acceptance  or  rejec¬ 
tion 

Members  of  the  review  team  should  be  selected  with  care  and 
should  be  thoroughly  briefed.  Particular  emphasis  must  be  placed 


Mi 


on  the  importance  of  unbi^edness  end  adherence  to  the  ranaom- 
nees  requirement  of  the  sampling  process  (e.g.,  in  oivier  of  task 
occurrence,  technician  assignment,  etc.}* 

It  is  moat  helpful  to  have  written  guidelines  for  the  review 
team,  including  the  necessary  data-collection  forms,  definitions, 
and  procedures,  &o  that  rigor  and  uniformity  are  established. 
Because  of  ft  natural  variability  in  observer  interpretation  and 
focus.  It  is  generally  advisable  to  have  at  least  two  review- 
team  uismbers  as  t  as,  k- time -data  recorders  for  each  sampled  task. 

8.6  DEMONSTRATION -DATA  FORMS  AND  RECORDS 

There  are  two  basic  forms  that  are  pertinent  to  the  demon¬ 
stration:  (l)  observer  record  of  individual  maintenance -task 
perf ormance,  and  (2)  summary  record  of  maintenance-task  perform¬ 
ance  . 


In  addition,  there  may  be  a  standard  form  for  maintenance- 
personnel  biographies,  a  checklist  for  review-team  members  to 
verify  that  all  requirements  are  being  met,  and  other  forms  to 
provide  supplement ary  information  or  meet  secondary  objectives. 

As  discussed  in  Section  III,  the  observation  of  maintenance 
design  or  procedural  deficiencies  is  often  an  Important  byproduct 
of  the  demonstration  tests, and  plans  should  be  made  to  identify 
areas  for  improvement. 

The  observer  record  of  maintenance -task  performance  is  the 
basic  data-collection  form  cn  which  actual  times  are  recorued 
by  the  review-team  member  for  each  maintenance  fce.sk.  The  summary 
record  Is  the  form  that  summarizes  the  task-time  information  from 
the  observer  records  for  use  in  analyzing  the  results  and  calcu¬ 
lating  the  decision  statistic  of  the  test. 

Because  the  content  and  design  of  such  forms  depends  on  the 
specific  conditions  of  the  test,  no  specific  form  is  recommended. 

A  sample  form  thd.t  can  be  used  as  a  guids  Is  presented  as  Exhibit 
3.  The  time  data  in  this  sample  are  recorded  by  the  running-clock 
method,  and  the  asjociated  actions  are  described  in  a  narrative 
manner.  After  completion  of  the  task,  each  action  can  be  coded 
according  to  preselected  categories  such  as  preparation,  fault 
location,  repair,  and  checkout.  A  debriefing  to  the  maintenance 
technicians  oy  the  review  team  is  recommended,  and  a  special  form 
can  be  prepared  for  this  purpose. 

The  summary  form  contains  general  background  information 
and  the  task  numbers  and  desired  time  elements  from  the  data  of 
the  individual  task- record  forms.  An  example  is  shown  as  Exhibit  4. 

An  example  of  a  personnel  data  sheet  (taken  from  the  Philco 
report***  )  used  in  previous  demonstrations  of  Air  Force  equipment 
Is  presented  as  Exhibit  5. 

x*  Xbid.  222 


I 


Contract  No 


Observer 

Test  Location  .  Site  y 
Maintenance  Personnel 


LOCATION  OF  FAULT 

Subsystem  _ 

Equipment  _ _ 

Assembly _  _ 

Part _ 


FAILURE  SIMULATION 

lype  of  Failure  _ 

Method  of*  Simulation _ . 

Observed  Symptoms _ _____ 

Operation  Mode  „ _ 


Time  (Minutes) 


Number  of 
Men 


Observer  Comments 


exhibit  3 

RECORD  OF  -MlNTAINABILITy. 


■DEMONSTRATION  TASK  TIME 


1 — 
Code 

— 

| 

Techni¬ 

cians 

(Codes) 


Observers 

(Codes) 


Task. Tinea  (Minutes) 

Prepa¬ 

ration 

Fault 

Location 

Repair 

Check- 

Out 

Total 

FEEDEMC3TSTHATICSJ  ACTIVITIES 


Fncr  to  the  demonstration,  the  review-team  director  is  re- 
©pcnsible  for  ensuring  that  all  necessary  preparations  for  the 
dsteonstration  have  been  completed.  The  Maintainability-Demonstra¬ 
tion  Plan  (see  Section  III)  should  be  used  »s  the  basic  reference 


The  major  tasks  are  described  below. 

6c7.1  Site  Visit 

The  site  should  be  visited  to  she  Cjk  Vjl®  existing  overall 
(mvironasTit  and  facilities  agalrss^  tia?o«  planned.  Site  personnel 
should  bft  briefed  on  the  character  and  purpose  of  the  demonstra¬ 
tion  to  ensure  understanding  and  cooperation.  The  arrangements 
made  for  equipment  delivery,  installation,  and  checkout  should 
also  be  made  as  applicable. 

The  arrsngecsents  made  for  the  necessary  maintenance  support 
material,  sucJi  as  test  equipment,  tools,  manuals,  and  spares, 
should  also  be  checked  against  prepared  checklists. 

Finally,  the  srrangaaents  made  for  the  demonstration-review 
terns  should  be  checked,  Including  the  observation  facilities, 
quarters  (for  both  the  aaihtenance  team  and  review  team),  instruc 
tional  guidelines,  data  forms,  and  stop  watches. 

8„7«2  Training  and  Indoc trinat ion 


Maintenance-team  and  review-team  training  and  indoctrination 
should  be  reviewed  to  ensure  that  all  important  details  have  been 
covered.  A  final  briefing  lust  prioz*  to  the  test  is  also  bene¬ 
ficial.  It  would  be  expeditious  to  provide  review-team  members 
at  this  time  with  a  checklist  of  items  for  verifying  that  the  test 
conditions  moot  the  requirsuents . 

The  maintenance  team  should  be  made  completely  aware  of  the 
purpose  of  the  test  and  the  procedures  to  be  followed.  It  should 
be  stressed  that  It  is  not  their  individual  performance  that  is 
being  tested  but  rattier  that  of  equipment  maintainability  design 
<®A  supporting  materials.  Before  the  test  begins,  the  review 
tern  should  obtain  biographies  of  the  maintenance  teem  to  aid  in 
task  assigcaaent. 

s.7.3  gMMMlBJtf 

When  the  test  sample  sise  has  been  established  and  the  main- 
population  identified,  it  is  necessary  to  determine 
the  actual  faults  to  be  induced  vib*a  sampling  is  accomplished  by 
fault  inducement .  (/or  a  natural  -failures  test,  this  effort 

be  directed  to  providing  the  information  necssscry  to  verify 
the  adequacy  of  the  observed  sampling  distribution  of  tasks.) 
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B5  us©  of  the  procedures  suggested  in  SvvZZm,  vt,  &  list  of 
maintenance  tasks  and  associated  mems  for  their  shots' 

b©  prepared.  It  is  recommended  that  sore  tasks  fchAn  r»«$uiss>£  S# 
prepared  In  order  to  anticipate  unexpected  difficulties  In  t&e 
task  simulation.  The  distribution  of  the  "backup *  teak  gisauld 
also  be  in  accordance  with  the  procedures  of  Section  YX. 


It  i*  suggested  that  trials  be  performed  tor  'She  various 
iau3t~selectio^  mstlioas?  to  ensure  that  the  system  indicot  lone 
will  be  as  planned. 

The  sequence  of  tasks  should  be  ordered  random If  (e.$.,  by 
a  "nun&srs  in  the  hat"  method}*  The  m&lnteaange  teotesiloiaas  cm 
then  be  assigned  to  the  tasas  in  order  except  when  each  sac  sry  -mb 
is  contraindicated.  Any  review-team  asfJgnrrat  tasvfd  also  be 
made  at  this  time. 


To  acclimate  the  technicians,  instill  confidence,  and  rwstace 
the  chances  for  unrepresentative  results  due  to  initial 
ness,  it  is  recommended  that  the  fifst  tusk  for  each  ttegnfti&Si 
be  designated  as  a  trial  task.  This  should  be  one  that  i*  well 
within  the  scope  of  his  training.  m&  experience  and  Will  hot  ii 
unusually  difficult.  The  results/  of  these  trial  tasks  Should 
not  be  included  in  the  analysis  $ut  should  be  handled  in  the  mm 
manner  as  all  following  tasks.  The  technicians  should  not  be 
informed  of  the  nature  of  these  tasks.. 


The  trial  tasks  also  accomplish  &  secondary  objective  -  ~ 
testing  the  review-team  procedures*  and  observer  data  fora*.  After 
a  trial  task  Is  performed,  the  review  team  should  ®e»t  to  identify 
and  correct  any  deficiencies  in  the  planned  pcrocedure®. 

The  result  of  this  effort  Is  a  listing  of  the  tasted  to  be 
observed.,  the  fawlt-’induceraent  means  for  thsir  sismalfifeim,  the 
order  of  their  occurrence,  and  the  assigned  technicians  am 
review-te.im  members. 

8.8  CONDUCT  OP  THE  TEST 

\  ,  y  _  *• 

This  subsection  is  concerned  primarily  with  the .  conduct  of 
tests  based  on  fault  Inducement,  sine©  the  natural-failure  samp¬ 
ling  approach  does  not  generally  require  or  permit  as  men  careful 
control . 

A  "typical"  step-by-step  procedure  for  conducting  a  fault- 
simulation  test  Is  as  follews: 

(l)  The  review-team  director  or  designated  represent stive 
ensures  that  the  t%*at  area  is  re&Sy  for  the  test  and 
vhat  necessary  observer  and  maintenance  personal  are 
available .  / 


{2}  11%  checklist  of  nec.*BQ*ycy  items  io  read  off  to  ensure 

that  all  necessary  supporting,  material  is  ts-vailabl* 
and  the  cnvirorssent  and  facilities  ere  satiafact  sy, 

(3)  The  equipment  (including  test  equipment)  is  checked  to 
determine  if  it  is  in  proper  working  order. 

(4)  She  first  of  the  preselected  malfunctions2*  is  inserted 
into  tise  oquipeaesit  in  accordance  with  the  prescribed 

~  procedure,  The  symptom-display  indications  may  be 
checked  at  this  tine. 

(3)  The  matetenanee  teohniciests  are  called  in  ettA  informed 
of  the  equipment  trouble  3n  a  manner  previously  agreed 
upon  end  as  stellar  as  possible  to  that  which  would 
occur  under  noraal-  field  operation.  This  may  include 
&  written  operator-comploilht  record  or  a  verbal  des¬ 
cription  by  &  designated  raaintenance  supervisor. 

(6)  Tine  recording  by  the  observers  in  accordance  with  the 
prescribed  task-time  record  starts  as  soon  as  actual 
maintenance  •  begins . 

(?)  Any  unusual  hindrance  or  delay,  such  as  unavailability 
of  a  required  spare  part;,  should  be  noted  by  the  ob¬ 
server®  so  that  final  analysis  will  provide  an  unbiased 
measure  of  active-maintenance  time  or  man-hours. 

(8)  When  th$  maintenance  task  is  completed  and  the  repair 
has  been  verified  by  the  review  teasn,  the  area  will  be 
cleared  and  preparation  for  another  task  will  begin. 

(9)  After  each  task  or  series  of  tasks,  the  maintenance 
team  should  be  debriefed  while  the  next  task  is  being 
prepared,  to  obtain  hralr  coaraents  on  the  maintainabil¬ 
ity  features  of  the  'fcquipoent  and  supporting  material. 

The  following  specific  precautions  should  be  taken: 

The  maintenance  technicians  must  not  witness  the  fault 

gasarfclon. 

•  Only  prescribed  maintenance  manuals  and  handbooks  should 
be  made  available  to  the  technicians. 


•  The  maintenance  technicians  should  not  be  permitted  to 
converse  with  contractor  representatives  during  the 
demonstration,  but  normal  supervisory  personnel  should 
co  made  available. 

•  A  careful  check  should  be  me£«  to  ensure  that  the  fault 
indue«K«nt  does  not  provide  abnormally  informative  clues. 

*eM  discussed  above,  it  is  recommended  that  for  each  new  tech- 
s&d m  or  teat,  &  trial  task  be  simulated  first, 

xf3ctK*  of  these  items,  in  slightly  different  fora,  appeared  In 
it  briefing  to  r®vlew-tea&  personnel  assigned  to  a  Navy  main- 
t«tnebility-^S«mK®str«tl<Ki  pro®?am. 


! I3Wa 
ime  r.tz 


•  The  a&itita&Mneet  technicians  should  mt  'mm  the  poswlbl* 
tasks  to  bo  si malited,  and  the  c&wt?actor  should  b® 
enawsa  *»  of  tbs  selected  teak  order. 

•  The  demonstration  site  should  provide  suffiei&sxfc 
visibility  for  adequate  review- tea*  ob&mu&ica. 

•  The  decision  to  include  repair  tia»  for  any  natural  ly 
occurring  failures  ox*  eecondery  failures  should  be  «£&» 
beforehand  in  case  such  events  occur. 

.  Data  records  should  be  reviewed  periodically  to  verily 
that  appropriate  detail  and  accuracy  are  being  recorded* 

One  specific  problem  that  can  occur  daring  the  test  is 
extremely  long  raalnienanca  tines.  Occasionally  &  fca  hnlciat  magr 
have  great  difficulty  in  diagnosing  a  failetfw  and  eft*.*  e  long 
period  of  tine  he  has  essentially  made  no  headway.  Vo  MLi 
this  problem,  f  t  is  advisable  to  designate  a  cutoff  tds***Mgr  *ss 
estimated  93th  percentile  value,  by  which  time  the  test  is  stopped 
unless  the  review  team  feels  that  a  productive  approach  is  under 
way. 

The  analysis  of  such  occurrences  poses  problems  sino®  the 
long  maintenance  times  may  well  be  due  to  ti^yperlance  of  the 
technicians ,  If  it  is  clear  that  with  experience  t£w  technician 
would  not  have  had  the  difficulty  (end  perhaps  thi&  ejaa  be  checked 
by  repeating  the  test  using  another  technician}*  an  estirwted 
value  of  the  expected  task  time  under  ©ore  realistic  ccaditloE* 
may  be  used.  As  an  alternative,  another  back-up  task  may  foe 
observed  to  replace  the  troubles  caw  one.  While  these  as*  prac¬ 
tical  solutions,  their  implementation  opens  up  the  question  of 
test  bias. 


In  case  a  review  of  the  pi^blem  indicates  that  the  fault  i® 
with  the  maintainability  design  rather  than  with  the  technician 
or  environment,  an  estimate  of  the  required  test  statistic  such 
as  the  mean  can  be  made  by  analysis  of  the  caaplet ©d-t«ak  times 
in  conjunction  with  statistical  theory  on  censored  or  truncated 
observations . 

As  a  simple  example,  assume  that  the  guidelines  to  the  review 
tea®  are  to  stop  the  teat  when  the  maintenance  time  exceeds  X* 

(say  an  estimated  99th  percentile  value).  If  a  observations  are 
made,  and  k  observations  are  greater  than  X*  (k  should,  of  course, 
be  small,  say  only  X  or  2),  then  actual  times  are  recv^ed  on 
(n~k)  tasks. 

If  a  lognormal  distribution  is  assumed,  the  maximum  likeJLl- 
h'xwS  estimation  for  the  «r*an  Is 


>.  is  <m  auxiliary  estimating  function  for  which  tab1  are 
available  r ia 


0.9  DATA  REDUCTION  AND  ANALYSIS 

The  primary  effort  (data  reduction  and  analysis}  is  the  develop 
menfc  of  tte  statistics  required  by  the  statistical  demonstration- 
teat  procedures,  such  as  mean  number  of  man-hours  to  complete 
corrective  maintenance  or  the  time  associated  with  the  95th 
percentile.  This  effort  should  be  under  Air  Force  control, 
preferably  through  the  review- team  director. 

Before  such  statistics  can  be  obtained,  however,  data  should 
fee  edited  to  ensure  recording  accuracy  and  consistency  and  to 
check  that  all  test-condition  requirements  have  been  met. 

The  assumptions  made  for  the  statistical-test  design  can 
first  be  checked,  and  if  there  is  indication  of  a  disparity, 
further  analysis  will  be  required  before  a  decision  can  be  made 
to  proceed.  An  alternate  nonparsmefcric  test  can  be  used,  for 
example,  if  it  is  found,  that  the  lognormal -distribution  assumption 
is  a  poor  one. 

In  addition  to  the  calculation  of  the  test  statistic,  further 
deta  are  generally  available  that  can  add  to  the  store  of  know¬ 
ledge  of  maintainability.  At  a  minimum  ,  these  data  should  be 
tabulated  and  Summarized. 


The  means  for  calculating  the  test  statistic  and  the  appli¬ 
cation  of  the  decision  criterion  has,  of  course,  been  determined 
beforehand,  and  once  numerical  values  are  obtained,  the  decision 
of  accepting  or  rejecting  is  fairly  straightforward. 


8.10  TEST  REPORTING 


The  final  effort  of  the  demonstration  activity  is  the  prep¬ 
aration  of  the  Maintainability  Demonstration  Report  by  the  review 
teasa.  For  extended  test  periods  interim  reports  may  have  also 
been  prepared . 


»s»  y  ’t'X’} 


The  final  report  need  not  be  unmces^arily  detailed#  a  luce 
reference  can  be  made  to  the  Maintainability  Demonstration  feat 

Plan*  The  following  is  a  possible  format  for  the  reports 

*  *■  ' 

1.  Introductory  Section 

4  *  , 

A.  summary  of  test  objectives#  including  identification  of 
equipment,  manufacturer,  contract  number,  numerical  require¬ 
ments,  demonstration  site,  and  review- team  members. 

2.  Test  Conditions 

A  summary  of  the  test  conditions,  including  maintenance 
personnel,  with  particular  reference  to  deviations  from  the 
Maintainability  Demonstration  Plan, 

4:  "•' £ 

3^  Test  Procedures 

1  ",  ,  .  .$  t 

A  brief  review  of  the  proceedings  of  the  demonstration  test, 
noting  particular  problems  and  means  taken  to  overcome  them. 

v 

4.  Test  Results  v* 

•  \ 

A  summary  of  the  observed  data  and  the  results  of  the 
analysis  made  for  decision  purposes.  p  ** 

5.  Discussion  ' 

A  discussion  of  the  best  results  and  analysis,  along  with 
qualitative  findings  of  the  review  team  and  maintenance  te«s» 
Deficiencies  in  test  design  and  procedures  should  be  noted 
here,  as  well  as  deficiencies  in  the  maintainability  design 
and  procedures  associated  with  tile  equipment  under  test. 

6.  Recommendations 

A  i pec if ic  recommendation  on  'acceptance  or  rejection  of  the 
equipment  under  test  and  ether  recommendations  for  improve¬ 
ment  in  equipment,  procedural  or  test  design  . 

8.11  ACTIONS  FOLLOWING  REJECTION 


It  is  a  requir-uaent  of  MIL- STD-471  that  a  retest  be  performed 
If  the  equipment  fails  to  pass  the  demonstration  test.  Such  a 
reteat  must  be  scheduled  to  permit  design  and  procedural  changes 
to  be  made  to  correct  existing  deficiencies.  Generally,  the  coot 
of  such  redesign  should  he  borne  by  the  contractor  and  not  be  a 
subject  of  renegotiation*10  If  this  is  not  stated  clearly  and  un¬ 
equivocally  .  in  the  initial  contract,  one  of  the  major  purposes  of 
demonstration  —  providing  the  Incentive  for  good  design  —  is 
thwarted. 


V8This,  of  course,  is  based  on  the  assisnpticn  that  test  failure 
Is  due  to  contractor  deficiencies  rather  than  Government  deficiencies 


A  more  direst  ^eans  for  providing  such  incentive  is  through 
m  incentive  contract  In  'which  the  amount  of  the  contract  award 
depends  on  the  results  of  the  demonstration  effort, 

Per  the  retest,  the  contractor  should  first  submit  &  report 
describing  the  efforts  made  to  improve  the  equipment 1  s  maintain¬ 
ability,  and  these  efforts  should  be  approved  by  the  procuring 
activity.  A  second  test  of  unimproved  equipment  or  procedures 
is  to  be  avoided. 

The  task-sasapling  scheme  should  be  revised  for  the  retest. 
The  same  procedure  may  be  applicable,  but  a  new  Bet  of  random 
numbers  should  be  selected  as  applicable  so  that  the  contractor 
cannot,  by  knowing  the  tasks  to  be  sampled,  plan  his  improve¬ 
ments  solely  to  pass  the  teat. 

All  other  conditions  concerning  the  conduct  of  the  test 
should  apply  for  the  retest. 


CONCLUSIONS  AND  RECOMMENDATIONS 


9.1  GENERAL 


This  concluding  section  is,  in  essence,  a  summary  of  the 
major  points  made  in  the  report.  The  major  conclusions  and 
recommendations  mads  in  the  previous  sections  are  repeated  or 
summarized  here  for  emphasis.  For  convenience*  the  pertinent 
sections  of  the  report  that  deal  with  the  subject  of  interest 
are  indicated. 


9.2  THE  CONCEPT  CF  MAINTAINABILITY  DEMONSTRATION  ' 

Maintainability  demonstration  must  be  a  contractual  require¬ 
ment  ‘in  accordance  with  MIL-STD-470,  and  pans  for  meeting  the 
requirements  of  MIL-STD-471  must  be  detailed  in  a  Maintainability 
Demonstration  Plan  submitted  by  the  contractor  (Section  1,1). 

The  demonstration  procedure  is  essentially  an  application 
of  the  statistical  theory  of  hypothesis  testing  and  must  be  plan¬ 
ned  and  conducted  as  such  (Section  1.1). 

9.3  THE  CURRENT  STATUS  OF  MAINTAINABILITY  DEMDNSTRATION 


Some  of  the  more  important  conclusions  and  recommendations 
resulting  from  the  survey  (Section  II)  are  as  follows: 

•  Maintainability  demonstration  is  primarily  a  main¬ 
tainability-control  and  information-generating  process, 

•  Many  contractual  documents  contain  inadequate  main¬ 
tainability-  demonstration  provisions . 

•  A  major  problem  concerning  the  conduct  of  the  test  is 
the  differences  between  test  and  field  environments. 


•  Allocation  from  higher-level  requirements  is  the  prefer?®** 
method  for  determining  numerical  test  requirements, 

*  The  mean  is  the  generally  preferred  index  of  spec ificat ion , 

■  MIL- STD -471  is  generally  acceptable  except  that  greater 
flexibility  should  be  offered. 

9.4  DEMONSTRATION  AS  A  PROGRAM  ELEMENT 

Timely  planning  and  careful  management  of  the  maintain- 
ability-demonstration  program  are  required  to  fulfill  the  require 
ments  of  MIL- STD-470  and  MIL-STD-471  (Section  3-1). 


The  following  additional  conclusions  are  made: 

•t  A  nseintainablllty-deraonstration  test  does  not  guarantee 
achieving  the  re.quired  aaintainabi? lty .  It  focuses  the 
contractor's  attention  on  maintainability,  hut  often  this 
is  not  sufficient  unless  penalties  for  test  failure  are 
included  in  the  contract  (Section  3.2). 

•  The  contractor's  maintainability-demonstration  plan  must 
meet  the  requirements  of  MIL-STD-471.  Its  development 
should  be  one  of  continuous  refinement  to  reflect  changes 
in  requirements  and  design  and  to  incorporate  the  results 
of  maintainability- design  reviews,  predictions,  and 
assessments  (Section  3.3). 

•  An  Important  by-product  of  a  maintainability-demonstration 
test  is  the  information  provided  for  improving  equipment 
reliability  and  maintainability .  T)ie  passing  of  the  test 
does  not  mean  that  important  improvements  cannot  be  made. 
The  procuring  activity  should  plan  to  have  appropriate 
personnel  monitoring  the  test  to  discover  design  defi¬ 
ciencies  and  recommend  improvement  (Section  3.4). 

9.5  THE  STATISTICAL  BASIS  OF  DEMD NSTRAT ION 

•  Full  understanding  of  the  meaning  of  the  a  and  p  risks 
associated  with  a  demonstration-test  specification  must 
precede  the  assignment  of*  numerical  values  (Section  4.2)." 

.  The  numerical  maintainability-demonstration  test  require¬ 
ments  must  be  presented  in  a  manner  not  subject  to  mis¬ 
interpretation.  This  is  especially  Important  when  the 
requirement  is  stated  in  terms  of  confidence  levels 
(Suction  4.3) „ 

•  The  choice  between  variables  t\nd  attributes  tests,  single, 
mitiple,  and  sequential  testf*,  nonparametric  and  para¬ 
metric  tests,  and  classical  and  Bayesian  tests  requires 
full  consideration  of  the  information  requirements  and 
necessary  assuiqptlons  associated  with  combinations  selec¬ 
ted  as  possible  alternatives  (Section  4.6). 

9-6  MA I NTA I NAB ILITY - DEM) NSTRAT ION-TEST  SPECIFICATION 


The  more  important  requisites  for  a  maintainabllity- 
demonstration-test  specification  are  as  follows 
(Section  5^1): 

■■  The  maintainability  index  should  represent  a  measure 
that  is  directly  Influenced  by  equipment  design  so 
that  the  producer  can  plan  for  high  assurances  of  a 
pass  decision,  but  bears  the  responsibility  for  a 
reject  decision. 


Relationships  (at  least  qualitative]  between  design 
parameters  and  the  maintainability  index  should  be 
Known  so  that  design  evaluations  and  predictions  sire 
possible. 

The  maintainability  index  should  be  appropriate  for, 
and  measurable  in-  the  demons trat ion- teso  environment 


The  maintainability  index  should  be  related  to  high 
level  system-requirement  parameters,  and  numerical 
values  should  be  consistent  with  values  for  these 
higher-level  parameters. 

Adequate  sampling  and  statistical-evaluation  pro¬ 
cedures  should  be  available  for  demonstrating  con¬ 
formance  to  the  requirement. 


Corrective  maintenance  is  generally  more  critical  than 
preventive  maintenance  when  operational  requirements  are 
considered,  especially  if  the  latter  can  be  scheduled 
during  non-use  periods  (Section  5-2). 

The  man- hour -rate  index,  such  as  maintenance  man-hours  per 
operating  houn  is  a  direct  function  oi  both  reliability 
and  maintainability;  therefore,  a  test  based  on  such  an 
Index  should  be  the  responsibility  of  both  the  reliability 
and  maintainability  groups.  Models  for  relating  expected 
man-hours  per  maintenance  action  to  man-hour  rate  can  be 
developed  so  that  a  test  based  on  the  former  provides  an 

indirect  control  on  the  latter  (Section  5-2). 

i 

The  mean  index  is  strongly  influenced  by  long  maintenance 
times,  while  the  median  is  not.  The  mean  generally  pro¬ 
vides  better  manpower  cost  corftrol,  is  derivable  from 
higher-level  specifications,  and  has  more  desirable  statis¬ 
tical  properties  than  the  nfedian.  The  median  is  applicable 
to  distribution-free  tests,  has  direct  operational  meaning 
in  terms  of  being  a  50-percent  percentile  value,  and  for 
the  iogncrmal  distribution  is  not  dependent  on  the  value 
of  a2  (Section  5.2). 

The  Maintainability  Index  Selection  Matrix  (Exhibit  2) 
should  be  used  as  a  guide  in  chosing  the  main  '.ainability 
index  (Section  5*2) 

Three  basic  criteria  for  assigning  numerical  values  for 
the  selected  maintainability  index  ore  (Section  5*3); 

(1)  The  specified  value  should  be  consistent  with 
higher  system -level  requirements. 


(2)  It  should  he  realistic. 

(3)  Xt  sh  ould  perwtin  to  the  demonstration  envirenaent. 

•  Trade-off  approaches  between  reliability  and  maintain¬ 
ability  Indices,  given  an  overall  availability  require¬ 
ment,  should  be  considered  for  obtaining  numerical 
requirements  (Section  5-3) • 

•  Maintainability  predictions  and  analysis  of  historical 
data  should  be  used  to  assess  the  realism  of  specified 
values.  Tables  XII,  XIII,  XIV,  and  XV  present  pertinent 
historical  data  (Section  5*3). 

•  Tiie  closer  the  ti*st  environment  to  the  expected  field 
’•’rtvironaent,  th  acre  meaningful  the  demonstration, 
jgvery  effort  she.  ild  be  made  to  achieve  such  similarity. 
Tables  XVI  and  XVII  present  information  factors  to  con¬ 
sider  and  causes  of  discrepancy,  respectively 
(Section  5.3) • 

•  Tli*  asajUpuaent  of  test  risks  must  consider  the  corres¬ 
ponding  effect,  on  required  test  saraple  size  (Section  5.4). 

•  Use  of  prior  information  in  terms  of  test  costs  and 
wrong-decision  costs  can  be  used  in  a  decision-theory 
model  for  assigning  appropriate  test  risks  (Section  5.4). 

9.7  SELECTION  OP  MAINTENANCE-TASK  SAMPLE 

•  The  natural-failure  approach  to  candle  selection  is  the 
preferred  choice  but  often  is  inpractical  because  of 
time  and  equipment  limitations  (Section  6.2). 

•  A  combination  of  natural-failure  and  fault- inducement  pro- 

'  cedures  should  be  considered  if  it  can  be  feasibly  labile- 

a&nfced  (Section  6.2). 

•  Fad  lures  that  occur  naturally  during  simulated  failure 
test  should  be  included  in  the  sanple  (Section  6.2). 

•  Care  must  be  taken  to  se©  tl»t  the  fault- Inducement  pro¬ 
cedure  generates  a  representative  sailing  of  aaintenance 
taaka.  Intermit tens ia c,  degradation- type  failures, 
steonf  ry  failure#,  am  the  like  should  be  considered 
along  with  the  usual  catastrophic -failure  modes 
(Section  6.3). 

* 

•  Has  contractor  should  make  plans  for  retaining  parts, 
circuit  cards,  a#eetf&li®s,  etc.,  that  have  been  rejected 
during  development,  reliability,  and  quality-control  tests 
far  u#e  ip  Inducing  noncataetrophic  failures  (Section  6.3). 
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A  list  of  aalcttansnoe  teak*  is  monamzy  £®r  te&L&mnttm 
natural  failure  test®  so  well  »*  Taul t -Induacnimt  teats 
(Section  6,3). 


*  Slagle  random  sampling  Is  preferred  over  proportional 
stratified  random  sampling  whe£  the  analytical  aspects  of 
the  demonstration  procedure  are  considered.  ^roporticaaal 
stratified  sampling,  however,  can  provide  better  assurance 
of  a  representative  sample  (See  Table  XX,  Section  6.4)* 

:  *x?7£k 

1  The  basic  criterion  ror  stratifying  the  population  <sf 
maintenance  tasks  is  the  expected  aalntename  tian  (or 
nan- hour 4;  that,  is,  tasks  within  a  stratum  should  rtkju: 
essentially  the  same  maintenance  effort  so  that  vmrleb; 
within  a  stratum  is  small  (Section  6.4). 

*  Tlie  frequency  wit  tv  which  tasks  are  selected  within  a 
stratum  is  a  function  of  the  relative  frequency  of  occur¬ 
rence  of  the  tasks  comprising  the  stratum  (Section  6.4). 

-  A  stratum  comprising  tasks  not  asso**  j& feed  srith  piece- 
part  failures  and  not  necessarily  &  result  ©f  unrelia¬ 
bility  should  be  developed  if  the  occurrence  probability 
of  such  tasks  is  not  negligible  (Section  6.4). 

*  The  synptem  matrix  can  be  used  as  an  approach  to  strati¬ 
fication,  especially  if  fault -location  time  is  the  major 
element  of  the  maintenance-task  times  (Section  6.4). 

STATISTICAL  MAimiF^ILITl'-DKJCNSTRATION-T^T  FLAKS 


i:  'M  «  'at."’ 

Si  •«* 


A  review  of  MCI  STD-471  plans  is  presented  in  Subsec¬ 
tion  7.2j  it  should  be  consulted  to  determine  the  applic¬ 
ability  of  these  plans  to  the  specific  problem  at  hand. 

Data  and  met  hods  for  estimating  »  the  variance  of 

the  logarithm  of  corrective-mintinance  time,  are  presented 
in  Subsection  7.3.5  for  use  in  determining  nuifierieal 
values  and  saapls-slze  requirements  and  for  use  in  Bayesian 
tests . 


Fourteen  non-Bayesi  an  m&lntainab  ill  fcy  ~  d  emona  tr  at  ion  plans 
axe  presented  in  Section.  7*3.  Alternatives  with  respect 
to  fixed  vs.  sequential  tests,  lognormal  vs.  nonparaactric 
testa,  and  various  parameter  specifications  are  considered 
Guidelines  for  test  select ion  are  also  presented.  These 
plans  are  believed  to  represent  improvements  over  the 
MTL“STD-4?1  piano  in  terms  of  greater  flexibility  in  risk 
&t£l^3msnt,  test-parameter  specification,  hypotheses,  and 
forms  of  testing. 


•  A  Bayesian  teat  la  developed  and  illustrated  in  Section  7.5 
for  a  f Jxed-sasple  test  based  on  a  lognormal  assumption 
for  the  distribution  of  maintenance  time.  Sampling-plan 
tables  and  aesihods  and  data  for  prior-distribution  analyses 
are  also  presented. 

9.9  T8&T  ADMINISTRATION  AND  IMPLEMENTATION 

•  The  scheduling  of  maintainability-demonstration  tests 
should  conform  to  the  test  categories  described  in 
4SR  So-l4  to  the  extent  possible  (Section  8.2). 

•  Management  should  make  a  continuing  effort  to  acquire  and 
evaluate  all  available  pertinent  information  applicable 
to  the  demonstration  (See  Table  XLV,  Section  8.3). 

_•  Air  Force  personnel  are  the  preferred  choice  for  main¬ 
tenance-team  selection.  Representativeness  with  respect 
to  education*  training*  skills*  and  experience  must  be 
considered  in  choosing  individuals  (Section  8.4). 

•  A  demonstration-review  team  unde*  the  control  of  an  Air 
Force  representative  should  be  established  (Section  8.5). 

•  Kxanple  data  forms*  presented  in  Section  8.6,  can  be  used 
as  a  guide  for  developing  forms  applicable  to  the  problem 
at  hand. 

•  PredsiBonstr&tlon  activities  by  the  review  team  should 
include  &■  site  visit,  training  and  indoc  trination  review* 
and  preparation  of  task  simple  package  (Section  8.7). 

•  For  each  technician,  a  trial  task  should  be  established 
for  acclimating  the  technician  and  instilling  confidence 
(3eetton  8.7). 

•  A  "typical"  step-by-step  procedure  for  conducting  the  test 
and  a  list  of  precautions  is  presented  in  Section  8.8. 

»  A  maintainability-demonstration  report  should  be  prepared 
by  the  review  teas.  A  e&sple  outline  is  presented  in 
Section  8.10. 

•  If  th®  equipment  fails  the  test,  a  retest  must  be  sched¬ 
uled  in  tfccovdanee  with  the  requirements  of  MXL-STD-471. 
The  task  sailing  for  the  retest  should  be  based  cry  a,  new 
set  of  random  makers  (Section  8.11). 


06*'# 


APPENDIX  A 

THE  LC3XCPML  DISffllBSfTIGff* 


1.  BASIC  PROPERTIES 

*"7  , 

A  random  variable  X  ha?  «  lognormal  probability  C^orribut* on 
if  the  logarithm  of  the  variable  is  distributed  normally.  The 
lognormal  probability  density  function  is 


where  AnX  *=  natural  logarithm  of  X* 

a  •»  *'  '  ,T1% 

If  Y  *  inX,  the  probability  density  of  Y  le  normal#  mean  e0 
variance  ff2. 

The  Important  moment  and  distributional  properties  of  the  log' 
normal  are  presented  below; 

2. 

Mean  ■  qO+g/ g 


Variance 


Median  X 


P  Percentile 


6  P  (2p  «  noraa.1  deviate,  i.e 


Coefficient  of  Variation 


Ja&glk  fry  Altchison  and  J.  Brown,  Cem- 
BSf  19^3#  Is  th>#  eof^plot©  reference  on 


vers; 


this  subject 


■i  r  ■  5  “  .  . 


v r>: 


..*?  _ : t _ si _ 


'^.'^  l  *?■„  "  •*, 

The  is  a  positively  skewed  distribution,  with 

dsgr®«  of  skmmess  increasing  ar.  c 2 increases.  From  the  above 


formulas,  it  is  seen  that  mode  <  median  <  Bean.  Because  of  the 


two  parameters  B  and  a2,  the  lognormal  is  a  relatively  flexible 
distribution.  Several  important  properties  possessed  by  the  dis- 


.  *  -  .  -  .  -  -  -  .  2, 


tribution  are  given  below.  The  notation  L  ($,a)  shall  be  used 


to  represent  a  lognormal  density  with  parameters  e  and  a 


2. 


If  2  -  L  l9fa2h  then  Z  «  cXb  »  (in©  be,  b2c2),  (c>9) 

'  X  z 


[case 


<*J 


(B)  If  2  *  L,  fe,02),  then  Z  -  1/x  -  \  (-e,c*5 


2, 


with  c»l#  b*-l  ] 


A 

(C)  If  2^  »  Lx  (©2*0^ 2)  and  Xg  =■  (02»a2  '  are  ind«P«ndent> 


h  ’  Vi-Ve  -  \<ai^V'i!a?+4,f>’  (ai>°^o) 


Properties  (A)  and  (B)  state  that  the  lognonnality  of  a 
random  variable  is  preserved  under  a  simple  shape  or  scale  trans¬ 
formation*  and  property.  {C}  states  that  the  product  and  quoti¬ 
ent  of  independent  lognormal  variants  is  also  lognormal. 


It  also  bs  shown  that  if  the  product  of  two  independent 
random  variables  is  lognormal,  then  each  random  variable  must 
slftp  bs  lognormal.  This  is  not  necessarily  true  for  dependent 
variates. 


The  following  central  limit  theorems  hold; 

Xf  Xj#  Xgr  • • • are  independent  positive  variates  with  tho 


mo*  distribution  and 
I  *  « 


j»l,  2,  ...,  n 


Var  Circe «  c2<»  2, 


» •  •  f 


n 


~T&. 


2  * 


•  - 


asymptotically  L  (ng,n©  ) 
« 


A-2 


As  a  corollary,  the  geopetrie  mean  {  If  XJ  is 

$-1  *A  f- 

asyaptotloally  lognormal,  mean  »0  variance  »  a  /n. 

(5)  If  X^,  Xg,  .,.,  Xn  are  independent  positive  variates  such 
that 

S(feX^)  -  0  j  <* 


Var(inX.)  -  a2  <« 


3  [jinX^-aJ2}  »©^<» 
then  Z  » TTXj  is  asymptotically  ^z^J^y 


provided  that 


-♦  0  as  n  «*  « 


•  )#&■ 

■Ml  *  !i-  J# 

-  'ft*  .  i  -  ^  • 

v'Vr<-,r.:.*  B 

•.-•<*■•>•.  St... *  i£  • 
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2.  DATA  ANALYSES 


2 . 1  Graphical  Analysis 

It  is  usually  advisable  when  analyzing  any  set  of  data  to 
plot  the  observed  sample  obssrvatioiie  so  that  distributional 
assumptions  may  be  verified. 

There  exists  logarithmic  probability  paper  which  yields  a 
straight  line  for  the  lognormal  cumulative  distribution.  Thus, 

If  the  cumulative  distribution  of  an  observed  sasqple  plots  ap¬ 
proximately  as  a  straight  line  >  the  lognormal  assumption  may  bi 
accepted  as  being  reasonable.  Naturally,  puch  a  test  is  not 
rigorous  but  affords  a  quick  method  of  judgirg,  Such  tests  as 
the  Chi  Square  test  and  the  Kolmogorov  Smirnov  test  are . preferred 
whan  more  rigor  is  required, 

t 

The  use  of  lognormal  probability  paper  also  affords  a  quick 


estimate  of  6  and  o‘ 
we  have 


From  the  quantile  equation 

*o.*6  -e9-* 

*0,50  "e# 
r  _  pG  +  0 

*0  ,84  “e 


G  +  Z  e 


To  obtain  estimates  of  X01g,  XQ  ana  XQ  a  straight 

liws  is  fitted  through  the  points  plotted  on  lognormal  probabil¬ 
ity  gaper  and  the  X  values  corresponding  to  the  cumulative  values 
of  lo  percent,  50  percent,  and  %  percent  are  read  off.  As  an 
example,  a  simulated  sample  of  50  from  a  lognormal  distribution 

with  &  «  3,  cF  **  0,5  resulted  in  the  observed  cumulative  distri¬ 
bution  plotted  in  Figure  A-l  on  lognormal  probability  paper. 

A  straight  line  fitted  through  the  points  yields  the  esti¬ 
mates  Xq#34  *  32*0,  Xq>50  *  20.5  and  X0l6  «  13*0.  Using  the 

above  equations  we  have  the  estimates  0  ■  3.02  and  a  •  0.^5 
corresponding  to  the  true  va3ues  of  3  and  0.5. 

2.2  Point  Estimates 


For  the  lognormal  distribution,  it  iB  advisable  to  consider 

- — •»— -  i  O 

separately  the  estimation  of  0  and  a  from  that  of  the  mean  E(X) 
and  the  variance  V(X).  Several  estimation  procedures  are  possi¬ 
ble  for  each,  such  as  maximum  likelihood,  method  of  moments,  method 
of  quantities,  and  graphical  approaches. 

Table  A-l  presents  these  various  methods  of  estimation  assum¬ 
ing  that  a  random  sample  of  n  observations  is  obtained.  The  cor¬ 
responding  variance  of  the  estimates  and  appropriate  comments  are 
also  given  in  the  table. 


To  illustrate  the  characteristics  of  the  possible  estimation 
procedure  a,,.  25  samples  of  50  observations  each  were  obtained  by 
computer  simulation  from  a  lognormal  distribution  with  C  *  3  and 

o2  tm  o.25t  Esblsatee  of  9,  a2,  E(X)  and  V(X)  were  computed  by  the 
mjdmxm  likelihood,  moments  and  quantile  methods  for  saoh  sample. 
The  average  of  these  sample  values  as  well  as  the  sample  standard 
deviation  of  these  ei»*"t®ate®  are  presented  in  Table  A-2.  It  is 
mm  fsfcftt  in  all  daces  the  estimates  are  reasonably  close  to  the 
true  value  but.  that  the  nulsHn-lltellhood  estimates  have  general¬ 
ly  the  greatest  precision  in  terms  of  minimum  standard  deviation. 


X  jo 


n*t*ov 


^trUnc*  of  latlaata,  V(») 


Ccanant  on  latlaata 


rtHiforrod  Mthod-alnlma  variance 

Inofflcltnt  ooap aral  to 

Acceptable  for  larga  n.  &uy  computation. 
Per  aarly  lJiilcatlon.  3ubj»ctlva. 

Preferred  Hotaod. 

Very  inefficient  eoaiArod  to  M.L.E. 
Aeoeptebla  for  largo  n. 

For  early  Indication. 

Preferred  method.  Moot  wwsj*.  vail  on. 

Acceptable  eepec tally  for  oonblnlnc 

•ample* . 

Aaoeptable,  minimum  efficiency-* 

•3  percent  m  e  2  -*  • . 

0on« rally  good  for  largo  n  and 
email  a- 

Preferred  method. 

/ 

Vary  lneffieient-ehouid  not  b*  used. 

Acoeptable-efflaiency  between  63  per- 
cant  and  70  percent. 

Ogne rally  good  for  large  n  and  ar  ill 


Inina  Livelihood 


Wiigl!— I  llttellb-ood 


'  1 

*t  -  ua.  |  k(.1—  ) 

^  *  Vj  ♦  i,lXt>.73, 

*4  *  *0.50 

pl  *  I  ‘“I  ’  V* 

X  Is 

*}  •  fa  i  -Uni 
*  n 

Sj  -  0.336  (fa  ^  M  -  fi^.oy) 

i,(x) .  e‘i  *„(**/» ) 

i^ix)  -  x 

*s<*>  -  e*3  ♦  **/» 

a  i  -2 

a4u)  -  e#* v  "4^ 

V*»  -  e“i  [«•„(*?)  -  wfi*  3)] 

vx,-i  \(h-*y 

-  e**  *  ei  (e*s  - 1) 

f4<x>  -  8*4  ♦  *4*  («**  -  l) 


me*  i-i 

rzsx  crnxAOiA  a»  teas  hk»«ttv*  yen  mumxams  op  thz  uxmcnuL  Dismwnc* 


(in)(X^  *V3  -  *1*  ♦  *») 


Not  Applicable 


2oV(n-l) 


K*4-*5**1)  , 

S.0«(o%^ 

Not  Applicable 


«*::  ♦  »2)  ^ 

zn 

y*a'/n 

[v  (;§)]•* 

Not  Applicable  Ny 

\ 

(y“  +  »y10  +  1»»‘  ♦  i»y*  ♦  »»*  )  »* 

hr<^)+(*»*-i)*T(:0]  °4 

Hot  Applicable 


TABLE  A -2 

AVERAGE  SAMPLE  ESTIMATE  (2)  AND  SAMPLE  STANDARD  DEVIATION 
OF  ESTIMATES  (S)  FOR  THE  FOIXQWINS  METHOD 


Parameter 

Theoretical 

Value 

Symbol 

Maximum 

Likelihood 

Moment 

Quantile 

9 

3 

E 

3.022 

3,026 

3.018 

S  ^ 

^  0.0626 

0.0633 

0.0659* 

2 

0 

0.25 

E 

0.242 

O.229 

0.240 

S 

0.0438 

0.0475 

0. 0455 

E(X) 

22.7 

E 

23.17 

23.15 

23.09 

S  - 

15.50 

15.54 

14.78 

V(X) 

147 

E 

146.2 

140.2 

146.3 

S 

38.16 

40.42 

40.07 

2.3  Confidence-Interval  Estimates 
For  9  and  a2 

p 

When  maximum  likelihood  estimates  of  6  and  a  are  used,  the 

A 

quantity  e-9  1b  distributed  as  t  with  (n-l)  degrees  of  freedom 

6/S 

and  is  distributed  aB  with  (n-l)  degrees  of  frac¬ 

as 

dcm.  Confidence -interval  estimates  are  as  follows: 

For  8,  a  p  percent  confidence  interval  is 

«'’*»  »  +  fcp,n-i 

where  t  -;i  is  the  p  percentile  of  the  t  distribution  with 
(n-l)  degrees  of  freedom. 

For  <? f  a  p  percent  confidence  interval  Is 
y^p^n~l  *  p2,n-ly 


where  i  is  the  p.th  percentile  of  the  distribution 

Pj^>n— j.  1 

with  (n  -1)  degrees  of  freedom  and  p.^  -  p2  =  jx 

For  the  moment  and  quantile  estimate a,  only  approximate  large- 
sample  confidence  intervals  may  be  obtained  using  the  central 
limit  theorem.  The  general  expression  for  the  (1  -  a)  percent 
confidence  interval  is 

E  *  ZW2’^ 

where  Z2-a^2  is  norma3-  deviate  corresponding  to  the  (l-a/2)th 
percentile. 

Table  A-l  gave  the  theoretical  values  for  V(e).  Since  large 
samples  are  assumed,  tne  estimates  §  and  o2  can  be  used  in  place 

O 

of  0  and  c  when  these  parameters  appear  in  the  equation  for  V(E). 
For  Mean  and  Variance 

Only  asymptotic  confidence  intervals  are  obtainable  for  the 
mean  and  variance.  The  sample  must  be  large  so  that  the  esti¬ 
mate  is  assumed  to  be  normal  with  mean  equal  to  the  characteristic 
being  estimated  (true  mean  or  true  variance)  and  variance  equal 
to  V(E).  Then  a.(l  -  a)  percent  interval  is  again  of  the  form 

s  ±  zW2  jm 

The  equations  for  V(E)  are  given  in  Table  A-l  and  again  0  and  fr2 

p 

are  used  instead  of  6  and  a  when  such  parameters  appear  in  the 
v(E)  equation. 
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APPENDIX  £ 

l«L’fIH*«RatHy!S3I0H  COMPUTER  PKIHTDOT3  PC® 
VARIAHCB  PREDICTION 


EQUATION  A(l):  VARIANCE  OP  l n ( ACTIVE -REPAIR  1=0®)  ..  ?1  B®JiF!tK*2S 


DATA  INPUT 


Relative 

Power 


Teat 

Concent 


Variance  of 
in  ART 
(°ain  AST) 

0.628849E  00 
0.452929E  00 
0.2332896  00 
0. 159264 E  01 
0.4173166  00 
0,2500006  00 
0-1034296  01 
3.9643246  >00 
0.1900966  00 
0.5670096  00 
3.2QT936E  00 
0.8010256  01 
0.5944416  00 
0.3748446  00 
0.1073306  01 

O. U88106  01 

P. 1258886  01 
321284c  00 

G.121000E  01 
0.6448096  00 
0.3457446  00 


Squ&ro  Soot 
of  Number  of 
Cbaervatiori 


0.1000002  ox 

0.1000006  Q1 
C.1000Q0E  31 
0.2000005  01 
0.2000006  01 
0.200000E  01 
0.3009006  01 
0. 2C0GC0E  01 
0.200030c  01 
0.200000':  01 
0.2000006  01 
0.200C002  01 
C. 1000006  01 

0.200000P  01 
0.310000E  01 
0.3000006  01 
0.2000006  01 
Oiiocoooe  oi 

0,3000006  Oi 
0.2CQ900E  Oi 
0.200000E  01 


0. 30C000E  01 
0. 3C0000E  01 
0.30000G6  01 
9.5G0C006  01 
t. 103003c  01 
0. 1G30QQ6  01 
0.2000006  01 
U. 1000006  01 
0.2C0GC0E  01 
0.1000006  01 
3.1000006  01 
0.2000006  01 
■tt.sooecot  oi 
o.icooooe  oi 
0.2000006  01 
0.2000 OOE  oi 
0.2000006  01 
0,2000006  01 
0.3000006  01 
0-3000006  01 
0.100G006  01 


0.2000006  01 
0.25000GE  01 
0.20000)6  01 
0,2500006  01 
0.200000E  01 
9.1000006  01 
0 ,-.3300006  C* 
0. 350G03S  OS 
0.2000036  Oi 
3. 300000E  01 
0. 3000006  C‘ 
C.200000E  01 
0.4000906  01 
0.300GC9E  0 1 
0. 20(10005  <?1 
0.2000006  01 
0-200000E  <»1 
0.400000E  01 
C.2000C0E  01 
0.200000E  01 
0.2000006  01 


0. 4000006  0| 
0.1414216  01 
0.4890426  01 
0.2828436  Oi 
0.3162286  Oi 
0.1732056  01 
0.591*036  01 
0.2000006  Oi 
0.7449496  01 
0.5196156  01 
0.3741*66  Oi 
0.3162286  01 
0.2449496  Oi 
0.1732056  oi 
0.31&2286  01 

Q.4K98986  Oi 
0.3316626  Cjl 
0.2645756  01 
0. 640 3 t 26  01 
0.47958 3E  0* 
0  28234R  01 


( continued) 


ROM 

cut. 

VALUE 

RON 

COL. 

VALUE 

ROW 

COL. 

VALUE 

2 

\ 

-0.09075 

3 

1 

-0.12240 

3 

2 

-0.22984 

4 

1 

0.61116 

4 

2 

0*42903 

4 

3 

0.00403 

REGRESSION  COEfFS. 

INDEX  VALUE 

0  -0.799E I9E  00  {Constant  term) 

1  0*3850826  00  (Coefficient  of  rp) 

2  0.221325E  00  (Coefficient  of  3D) 

3  0.116839E  00  (Coefficient  of  TC) 

PARTIAL  CIKR.  CUEFFS. ,  STD.  DEV.  £  T  i  OR  *1 

1  0.7540E  00  0.81376-0]  0.4?33E  01 

2  0*66286  00  0.60646-01  0.3t.60r  01 

3  0,32?5£  00  0.8l?5E-Q!  0.1*2?  01 

ft  SQUARED  IS  0.65271c  a  IS  C. 8-  79C9 
STD,  ERROR  IS  G.25590E  00  STO.  ERROR  SQD .  IS  0.65484c~n; 

F  IS  0.1065046  02 


gauss  nult : pi  teas 

ROW  COL.  VALUE  ROW  COL.  VALUE  *PH  COL.  VALUE 

1  l  O.lOiiE  OP  i  2  0.92  V  l-02  1  j  0 . 1 5 1 1 - ; )  1 

2  2  0.56156-0'  2  3  •>.  1 8-.5C-  i  3  3  U.iO^lE  00 


(continued) 


VARIABLE  NO.  MEAN  STD.  DEV. 


1  2. 071436  00  '/•  13975E-01 

2  2.12956F  00  9.77026E-01 

3  2.456721:  00  7.27199E-01 

4  7.4621HE-QI  4. 003466-01 


CORRELATION  MATRIX 


Equation  A(l)  (continued) 


Equation  A(i)  (continued) 


/ 

/ 

TEST  INDEPENDENT  VARIABLES  FOR  SIGNIFICANCE 


i 

ST 

VAR. IS 

1 

R»«2 

«  0,374249 

F 

•11.364 

GRIT.  VALUE 

IS 

1.760 

2 

ND 

VAR. IS 

2 

R®»2 

*  0.610986 

F 

"10.9*4 

GRIT.  VALUE 

IS 

1.77C 

1 

RD 

VAR. IS 

3 

R*»2 

«  0.662716 

F 

■  2.0*3 

CP  IT.  VALUE 

IS 

i.rto 

ALL  VARIABLES  SIGNIFICANT.  SELECTION  ORDER  IS  123 


ACTUAL  VS.  predicted  results 


OBSERVATION 

ACTUAL 

PREDICTED 

DEVIATION  WEIGHTED  DEV. 

1 

0.62BB5E 

00 

0.43292E 

00 

0.14593E  00 

0.16902E  90 

2 

0.45293E 

00 

0.541346 

00 

-0.88408E— Ql 

-0.362026-01 

3 

0.23329E 

GO 

0.48292E 

00 

-0.24963E  00 

— 0. 33903E  00 

4 

0.1502ftE 

01 

0.13S91E 

01 

0.22357E  00 

0.18310E  00 

5 

0.M732E 

CO 

0.425356 

00 

-0.80328E-02 

-0.73552E-02 

6 

0.2D000E 

00 

C. 3085  IF 

00 

-0.585092-01 

-0.293446-01 

7 

0.103436 

01 

0. 120796 

01 

-P.IT272E  00 

-0.29538F  CO 

8 

0.96432E 

00 

0.600616 

00 

0.363726  00 

0.21063E  00 

9 

0.19010E 

00 

0.646676 

00 

-0.45658E  DO 

-0.323836  00 

10 

0.56701E 

00 

Q.54219E 

00 

0.24S21E-01 

0.37345E-P1 

11 

0.207946 

00 

0.54219E 

00 

-0.33425E  00 

-0.362J3E  00 

12 

0.80103F. 

00 

0.646676 

00 

0.15435c  00 

0.I6133E  no 

13 

0.59444E 

00 

0.49527E 

oc 

0.99170E-'0l 

0.70337E-P1 

14 

0.87984E 

00 

0.54219F 

00 

0.33766E  00 

0.169346  00 

15 

0.10733E 

01 

0  < 1031 3E 

01 

0.41544E-01 

0.3804PE-01 

16 

0.11S81E 

01 

*'»  103.18E 

01 

0.15634E  00 

0.2217EE  00 

17 

0.12589E 

01 

0.64667E 

00 

0.612216  00 

0.58792E  00 

18 

0.52128E 

00 

0.49527E 

00 

0.26013E-01 

0.1993?C-01 

19 

0.12100E 

01 

Q.12531E 

01 

-0.43081E-01 

-0. 79874E-01 

20 

Q.644GIE 

00 

0.868COE 

00 

-0.22319F  00 

-0.30993E  00 

21 

0.34574E 

00 

0.42535E 

OP 

-0.79605E-01 

-0.652956-01 

S70.  DEV.  OF 

OEV.  IS  0.1 

13770E 

00, VAR. 

IS 

D.18961E-01 

A/G.  DEV . 

IS  - 

0.37982E-* 

07 

EQUATION  A(2):  VARIANCE  OF  in ( AGTXVE~R£PAIR  TIME)  ~  13  EQUIPMENTS 


DATA  INPUT 


Maintenance 

Coanlexity 

T1®) 

Relative 

Power 

(RP) 

Efficiency  of 
Information 
Transmission 
(ET) 

Variance  cf 
in  ART 
(o£&n  ART) 

Square  Rcov, 
of  Number  of 
Observations 

Oo 120000E 

02 

0.100000E 

01 

G* 2C1000E  00 

0.452929E 

00 

0.14  14216 

01 

0.160000c 

02 

0.  1000006 

01 

0.479000E  00 

0.233289E 

00 

0*4690426 

•J 1 

0.180000:6 

02 

0.200009E 

01 

0. 156000E  00 

0.417316E 

00 

0.3162286 

01 

0.14C000E 

02 

G. 2000006 

01 

0.7900006-01 

Or  964324E 

00 

0.2000006 

u 

0.12J000E 

02 

Q.200000E 

01 

0.4720006  00 

0.190094c 

00 

0 • 2449496 

u  i 

0.140000E 

02 

0.200000E 

01 

0.460000E  00 

0. 567009E 

00 

0. 5lOol66 

01 

Or 120000E 

02 

0.20000QE 

01 

0.477000E  00 

Q.207936E 

00 

0.3741666 

01 

0.14000QE 

02 

0.200000E 

01 

0.485000E  00 

0.S01025E 

00 

0.3 16228E 

01 

O.2SCO0OE 

02 

O.IOGGOOE 

01 

0.438000E  00 

0.59444 IE 

CO 

0. 2449496 

0* 

0.17C000E 

02 

0.20G09QE 

01 

0. 8400002-01 

0.879844E 

00 

0*1732066 

9  A 

C. 310000E 

02 

0«300000t 

01 

0. 327000E  00 

0.121000E 

01 

0.6403126 

01 

0.230000E 

02 

0.200000E 

01 

0.482000E  00 

0.A44809E 

00 

0.4795036 

01 

0.240000E 

92 

0.2CC000E 

01 

0. 7220Q0E  00 

0.345744E 

00 

0.232«436 

01 

(continued) 


/ 

/ 


/ 

/ 

/ 

/ 

/ 


Equation  A(2)  (continued) 


VA8JA3LE  HO. 


VALUE 


VALUE 


VALUE 


1  0,52042 


1  0.01915 


2  “0. 16006 


1  0.65104 


0.68856 


3  “0.42610 


REGRESSION  COFFFS. 
INOEX  VALUE 
0  0.365817E-01 

*  0.233J J2E-01 

2  0.224846E  00 


(Constant  Term) 
(Coefficient  of  !*) 
(Coefficient  of  Rp) 
(coefficient  of  ET) 

T  FOR  R1 
0.2WE  01 
0.1879E  01 
0. 3T49E  00  -0.2082E  01 

IS  0.851106 

ST0»  ERROR  S90.  IS  0.439771 
O.788450E  01 


0.1061E-01 


0.5308E  00 
-P.5702E  00 

SQUARED  IS  0.724381  * 

'•  is  0.20971E  00 

f  IS 


GAUS|  MULTIPLIERS 


VALUE 


VALUE 


5.2560E-02 
0.3256E  00 


(continued \ 


_____ 


Equation  A{2)  (continued) 

T£ST  INDEPENDENT  VARIABLES  FOR  SIGNIFICANCE 


s 

5T 

VAR* IS 

i 

R»*2 

*  0. 47*1 19 

F 

«  9.917 

CRIT. 

VALUE 

IS 

1  660 

2 

NO 

VAR* IS 

1 

R#'*2 

-  0*591614 

F 

*  2.877 

CRIT. 

VALUE 

IS 

1.880 

3 

RO 

VAR. IS 

3 

R«e2 

»  0.T24381 

F 

4.335 

CR!T. 

VALUE 

IS 

1.910 

ALL  VARIABLES  SIGNIFICANT.  SELECTION  ORDER  IS  213 


actual  vs.  predicted  results 


OBSERVATION 

ACTUAL 

PREDICTED 

DEVIATION  WEIGHTED  DEV. 

1 

0.A5293E 

00 

0.J6344E 

00 

0.69490E-01 

0.29G19K- 

-i'  l 

2 

0.23329E 

00 

0.25851E 

00 

-0. 252235-01 

-0. 34935E* 

■01 

3 

0.41732E 

00 

0.78346c 

00 

-0.36616E  00 

-C.34191C 

00 

4 

0.96432E 

00 

0.75066E 

00 

0.21367E  00 

0.12618E 

00 

5 

Q.1901QE 

or 

0.3956QE 

00 

-0.20550E  00 

-0.14864b 

00 

6 

0. 56701 E 

00 

0.45164c 

DO 

0.11537E  00 

0.1770IE 

00 

7 

0.2G794E 

00 

0.39168E 

00 

-Q„ 18374E  00 

-0.203005 

00 

8 

0.80103E 

00 

0.43202b 

00 

0.36900E  00 

0*  34456t 

00 

9 

0.59444E 

Oo 

0.50051E 

00 

0.93932E-01 

0.67941c- 

■01 

10 

0.87984E 

00 

0.81667b 

00 

0.63170E-01 

0.3230F 

01 

11 

0* 12100E 

01 

0.11772E 

01 

0.32805E-01 

0.62C2' 

.11 

12 

0.64481E 

GO 

0.64470E 

00 

0.6122 3E-03 

0.86700E- 

■03 

13 

0.34574E 

00 

0.47915E 

00 

-0.13341E  00 

-0.1X142E 

00 

STO.  OFV.  OF 

DEV*  IS  0.1 

U395E 

00, VAR. 

IS 

0, 12986E-01 

AVG.  OEV* 

IS  - 

0.11514E-07 

EQUATION  B:  VARIANCE  OP  INACTIVE  MAN-HOURS) 


21  EQUIPMENTS 


DATA  INPUT 


Relative 

Power 


Signal  Dfta 
Handling 
(SB) 

C.300000E  01 
0*  300000E  01 
0. 3000001  01 
0.500000E  01 
0.190000k  01 
0. IQOOOOfc  01 
0. 200000k  01 
0.100000k  01 
0.200000E  01 
0. 100000k  01 
0. 1QOOOGE  01 
0.200000E  Cl 
0.200000E  OS 
0. 100Q00E  01. 
0.200000k  0) 
0.200000E  01 
0.200000c  01 

0.20  )000E  01 
O.JOOOOOt  01 
Qo3Q0Q00E  01 
0.100000E  01 


Taat 

Concept 


Variance  of 
ki  MH 
(aaAn  MH) 

0.6806255  00 
0.13525TE  01 
0.3646165  00 
0.149573E  01 
0c546l22E  00 
0.425104K  00 
0.1597T0E  01 
0. 155C03E  01 
0.378225E  00 
0.73273AE  00 
0.8026165  00 
0.105268E  O' 
0.594441E  00 
C.75J424E  00 
C.154008E  01 
0. 163840E  01 
0»  1555015  01 
0.412H4E  00 
0. 12701 JE  91 
Os  1094 J  2E  01 
0.6006255  00 


Square  R*ot 
of  Kuatoer  of 
Observations 

o.<ooooot  oi 
0.141421E  01 
0.469042c  01 
0.28284?E  01 
Oq  3162285  01 
O.U3205t  01 
0.591408E  01 
C.2P0000E  01 
0.244949E  01 
0.5196155  0l 
0.774J66E  01 
0.316228E  G1 
0.2V4949E  01 
O.I72205E  01 
0.316228E  01 
0.48?898E  01 
0.331662?  n 
0.2645T4E  01 
0.640312E  01 
0.479383E  01 
0.2928435  01 


9.100000E  01 
0.100000E  01 
0a 100000E  01 
0.290000E  01 
0.200000E  01 
0.200000E  01 
0. 300000k  01 
0.200000E  01 
0.200000E  01 
0.200000E  01 
9.20OOOPE  01 
O.ioOGQOE  01 
Q*1COOOOC  01 
0.200000E  01 
C.30Q000E  0» 
0.300000E  01 
0.20000QE  01 
0.100000E  01 
0.3QOOOOE  01 
0.200Q00E  01 
0.200000E  01 


0.200000E  01 
0.230000E  01 
0.200000E  01 
0.250000E  01 
9.200000E  01 
0.100000E  01 
0.350000E  01 
0. 350000E  01 
0.  200000E  01 
0.300000E  01 
0.300000E  01 
0.200000E  01 
9.400QCQE  01 
0.300G00E  01 
0.20000CE  01 
Q.20Q0U0E  01 
0.200000E  01 
0.4000005  0-1- 
0.200000E  01 
0.200000E  01 
0.200000E  01 


(continued) 


(Equation  B  (continued) 


VARIABLE  NO 


MEAN 


2.  Q? 1^3E  00 
2.U956E  00 
2.45o726  00 
l.of54lt  OG 


7.139?of:“01 
0. 77G26E-01 
7.271996-01 
4.64 141E-9 1 


CORRELATION  natri X 


ROW  COL.  VALUE 


HOW  COL.  VALUE 


2  1  -0.09075 


3  i  -0.12240 


3  2  -0.22984 


1  0.71304 


4  2  0.23930 


REGRESSION  CCEFFS. 

INDEX  VALUE 

0  -0. 711014E  00  (Constant  term) 

1  0.501373E  00  (Coefficient  of  PP) 

2  0.169711E  00  (Coefficient  of  SD) 

3  0.132883E  00  (Coefficient  of  TC) 

PARTIAL  CORR.  CCEFFS.,  STD.  DEV.  6  T  FOR  A 


0.5230E  Q\ 


0.49936  CO 


0.96306-01 


0.13806  01 


R  SQUARED  IS  0.64173a  ft  IS  0,801084 
S?0.  ERROR  IS  3.30146E  00  STO.  ERROR  SCO.  IS  0.90373':-0l 

F  IS  0.1015G3E  02 


GAUSS  NULi I PLIERS 


ROW  COL.  VALUF 


ROW  COL.  VALUE 


OL.  VALUE 


1  l  O.iOllE  00 

2  2  0.561^-01 


1  3  3.15016-01 
3  3  0.10216  00 


(continued) 


Squat Ion  B  (continued) 

TEST  INDEPENDENT  VARKAI&FS  FOfc  SIGNIFICANCE 

I s  il  1 :::! :  i:El ' SR:  &s 

*  «u  vARaiS  3  *  G«64£?36  f  •  }t9Q4  CRST#  VALV£ 

AU.  VARIABLES  SIGNIFICANT*  SELECTION  OftOg£  f$  i  2  3 


ACTUAL  VS*  PREDICTED  RESULTS 

ACTUAL  PREDICTED  DEVIATION  WEIGHTED  CEV 

&B063L  00  0. 5&526E  00 

I3526E  M  0.6MTQE  SO 

364 82E  00  0.56526E  00 

16957E  01  0«1h?25E  0! 

5AUI2E  00  0. ?272l£  UO 

A25IOE  00  0.59433E  00 

15977E  01  0*  X5976E  01 

15500E  01  0.92664E  00 

37823E  00  0.89692E  00 

73274c  00  0.86009E  00 

B0282E  00  0. 860096  00 

10527E  01  0.B9692E  00 

5944*C  00  0.66132E  00 

T5M2S  00  Q.360C9E  00 

1 540 IE  01  0, S3983E  01 

•03B4E  01  9.  13953E  01 

i5^50fc  01  0.89692E  00 

U216E  00  0.661326  00 
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maintainability  goals  during  system  development  end  design.  Volume  I  contains  the 
results  of  •  study  to  develop  improved  maintainability  prediction  techniques  for  use  on 
all  major  classes  of  Air  Force  electronic  systems  at  the  equipment  level  of  maintenance, 
Tha  techniques  were  developed  for  application  from  concept  formulation  through  the 
detail  design  phases  of  system  development.  A  data  collection  program  was  conducted  +o 
acquire  detailed  information  on  maintenance  actions  perfo.ne^.  in  the  field.  The  data 
were  analysed  to  determine  relationships  between  aeiu<,ensuce  design  variables  and 
maintenance  time.  Prediction  models  were  developed  for  corrective  and  preventive  main¬ 
tenance.  Predictive  v^Ietiooshipc  between  maintenance  time  and  maintenance  manhou-  3 
were  also  developed.  A  data  base  was  established  for  uce  in  future  investigation  of 
aaintdnaoi lity .  (  ■  * _ 

Maintainability  ftpsoastration  is  a  testing  procedure  for  assuring  the  acquisition  of 
equipment  and  systems  'utoieh  meet  specified  numerical  maintainability  requirements. 
Volume  II  contains  the  results  of  a  study  to  develop  improved  aaintainablllty  demon¬ 
stration  procedures  for  all  major  classes  of  Air  Force  equipment.  An  industry  and 
government-wide  surrey  /as  conducted  to  provide  insight  into  the  current  status  of 
maintainability  demonstration  and  to  ini  late  research  into  the  technical  and  adminis¬ 
trative  aspects  of  demonstration.  Specific  recommendations  nad  guidelines  were  de¬ 
veloped  ter  maintainability  Index  selection,  maintenance  task  sampling  procedures, 
statistical  maintainability  demonstration  pleas,  and  for  test  planning  and  administra¬ 
tion.  The  use  of  prior  Information  for  specifying  humberical  requir ^msnts ,  designing 
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